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ABSTRACT 

In 1988 r a survey was conducted of irember 
institutions of the American Association of Community and Junior 
Colleges (AACJC) . The survey was designed to study and advance the 
use of student outcomes measures for assessing institutional 
ffectiveness in two-year institutions. Three sets of student outcomes 
were identified for In-depth study: academic progress and employment 
ouucomesr student learning outcomes r and student satisfaction 
oi^tcomes. Responses were received from 675 institutions r representing 
approximately 54% of the total AACJC membership. Major findings of 
the survey were as follows: (1) 61% of the colleges used academic 
progress and employment outcomes measures to assess institutional 
effectiveness, and 66% of the colleges gave higher or far higher 
priority to these outcomes than to other types of student outcomes; 
(2) only 35% of the colleges measured student learning skills 
outcomes and used them for assessing institutional effectiveness; (3) 
skills assessment at entry was more common than exit-only assessment 
or entry-exit comparisons; (4) 55% of the institutions used student 
satisfaction to assess institutional effectiveness; (5) about 75% 
used measures of academic progress and employment outcomes in the 
accreditation process; (6) curriculum development was most often 
cited as the activity most affectel by the use of outcomes measures; 
(7) respondents expected the priority associated with the use of 
student outcomes to increase within the next 3 to 5 years; and (8) 
enrollment size, accreditation affiliation , and geographical setting 
were not significantly related to the use of outcomes measures. The 
survey instrument with percentage responses is included. (JHC) 
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BACKGROUND 



"The Governor is certainly pushing assessment and student 
outcomes in community colleges as part of his accountability 
move in higher education." 

"Yeah, I bet he will soon tie some of our funding to the 
results." 

"Well, if he does the community colleges are certainly in 
better shape than the four-year colleges because we have 
been doing assessment for years!" 

Have you been involved in or overheard conversations like this? Many 
of us involved in the two-year college business have experienced such 
encounters. But, how much assessment have we done? What are the results? 
Are we in "better shape" than our four-year colleagues? What would happen 
if part of our funding was dependent upon the results of our assessment 
program? 

Questions such as these prompte- AACJC officials to request of its 
affiliate, the National Council of Instructional Administrators (NCIA), to 
consider and tudy the issue of assessment in the community college, and to 
develop an NCIA position statement on "value-added" instruction. Under then 
president Carol Viola, a task force was formed and I was asked to chair the 
group. 

After one informal meeting of some of the group attending the 1987 
AACJC conference, I contacted the American College Testing Program (ACT) 
to determine their interest in participating in and supporting the project. This 
initial contact resulted in a series of telephone calls, letters and meetings which, 
by December 1987, culminated in PROJECT COOPERATION. 

PROJECT COOPERATION is an effort of NCIA, NCSD (National Council 
of Student Development), and ACT to engage cooperatively in a study of 
issues associated with institutional effectiveness through outcomes assessment. 
The central thrust of the project is: 
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- To acquire, organize and disseminate information pertinent to current 
practices by community colleges related to the assessment of 
institutional effectiveness. 

- To develop among AACJC member institutions awareness and 
understanding of the issues and challenges associated with assessing 
and improving institutional effectiveness. 

- To develop a statement that identifies and promotes implementation 
and evaluation of "high potential" models for determining institutional 
effectiveness, with emphasis on instruction and assessment aimed at 
developing students' general education skills, in particular, and student 
support services, in general. 

- To design and carry out studies for implementing and evaluating 
these "high potential" models and for documenting and disseminating 
information resulting from the studies. 

- To inform and train NCSD and NCIA members in means of achieving 
institutional effectiveness through outcomes assessment based on the 
efforts described above. 

This document is the most comprehensive snapshot of what community 
colleges are doing about institutional effectiveness through outcomes, and what 
areas representatives of those institutions intend to move into in the coming 
years. The uses of this document should be many as community college 
faculty, administrators, and board member consider the status of their 
respective institutions compared to others and begin to plan effectively to 
adopt an institutional effectiveness program that is unique to the institution. 

There are a number of people that deserve acknowledgment. And, at the 
risk of forgetting someone, I express my appreciation to the original task force 
members, the small committee that met in Kansas City on a cold February day 
to begin to design the survey instrument, to Carol Viola and Betty Duvall of 
NCIA, to Robert Keys and Walter Bumphus of NCSD, to Andrew Falcone and 
Johnnie McClinton of my staff, to ACT staff Don Carstensen, Richard Rowray, 
and David Lutz. Finally, to Susan Cowart of ACT who wrote the following 
analysis of the survey results. Her talents, persistence, and determination are 
sincerely appreciated. 



Wayne E. Giles 
Kansas City 
January, 1990 
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EXECUTIVE SUMMARY 



This report is based on responses from 675 institutions, and that represents 
about 54% of the total institutional memberships by two-year institutions in the 
American Association of Community and Junior Colleges (AACJC). The survey 
is part of a collaborative effort by the National Council of Instructional 
Administrators, the National Council for Student Development, and American 
College Testing to study and advance the use of student outcomes measures 
for assessing institutional effectiveness in two-year institutions. 

Three sets of student outcomes are identified for in-depth study in terms 
of the importance associated with their use for assessing institutional 
effectiveness, actual use of items in the sets of outcomes, priority associated 
with their use now and in the future, and satisfaction with that use. Further, 
the question of how these student outcomes measures feed back into the 
operation of the institution by impacting activities and functions is explored. 
And, finally, the way institutions are organized for assessing student outcomes 
and institutional effectiveness is analyzed. 

Academic Progress and Employment Outcomes 

Academic progress and employment outcomes measures are used to assess 
institutional effectiveness by well over half (61%) of the institutions represented 
in the survey. The average for importance ratings for individuals across all 21 
items is 70%, and the average \xsv rate is only 37% of the items. Two-thirds 
of the institutions give higher or far higher priority to these outcomes for 
assessing institutional effectiveness than to other types of student outcomes. 
The number giving far higher priority within the next 3-5 years is expected to 
double. Finally, respondents generally sense moderate satisfaction at their 
colleges with the use of academic progress and student outcomes measures for 
assessing institutional effectiveness. 

Student Learning Outcomes 

Sixteen different learning skills are included in the set of student learning 
outcomes measures. Only slightly over one-third (35%) of the institutions 
measure student learning skills outcomes and use these for assessing 
institutional effectiveness. The average use rate is only 20%, and nearly one- 
half of the institutions use none of the items included in this list. On the 
other hand, the average score on the importance index is 83%, and only 3% of 
the respondents rating the importance of these items at 50% or less. Process 
skills are rated as most important and least used, relative to foundation skills 
(most used) and general competency type measures. Entry-only assessment is 
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more common than exit-only or entry-exit comparison assessment, but exit 
assessments do exceed entry assessments for the less used process skills and 
general competency measures. Foundation skills are most often perceived to 
be most important at present, but competency in general education or field of 
study is expected to be perceived as most important within the near future, 
i.e., next 3-5 years. The number of institutions that expect to give far higher 
priority to these outcomes in the near future is three times that currently giving 
to priority, and the number giving high or far higher priority win 
double within the next 3-5 years. Sutisfaction with the use of student learning 
outcomes to assess institutional effecttveness is on the lower side of medium 
satisfaction. The differences between mean satisfaction scores for student 
learning skills and the other two student outcomes measures is statistically 
significant at the .000 levtl. ' 

Student Satisfaction Qutr^ mpg 

Virtually ail of the respondents, over 99%, award at least 50% of the 
importance points to the 11 items included in student satisfaction outcomes, 
but one-third of the institutions use none of these items to assess institutional 
effectiveness, and half of the institutions use no more than four of the 11 items 
for that purpose. Overall, 55% of the institutions use student satisfaction 
measures to assess institutional effectiveness. 

The three items currently perceived to be the most important student 
satisfaction measures for assessing institutional effectiveness are also expected 
to be most important within the next 3-5 years. The level of priority associated 
with the use of student satisfaction outcomes is expected to increase, but the 
magnitude of change is not as great as that expected for the other two sets of 
outcomes measures examined previously. Satisfaction with the use of these 
measures for assessing institutional effectiveness is at the medium level, and 
tiiat is about equal to the satisfaction score for academic progress and 
employment outcomes, and the difference in these mean scores is not 
statistically significant. 

Uses of O utcomes Assessment Information 

Twenty-five activities and functions lypical to the institutions in the survey 
are listed on the questionnaire and respondents were asked to indic^ite which 
sets of student outcomes affect each of the activities and functions. 
Accreditation studies are most often impacted by the use of these three sets of 
outcomes measures. About three-quarters of the institutions use measures of 
academic progress and employment outcomes in this process, over half use 
student learning outcomes, and nearly 60% use student satisfaction measures 
for this purpose. These outcomes are used on average by 60% of the 
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iiistitutions in institutional planning, and a:^ average of 52% use these in 
curricului development. 

When respondents were asked to select the three activities and functions 
most affected by the use of these outcomes measures, however, curriculum 
development received the most mentions followed by institutional planning and 
course offerings. Accreditation studies rank fourth in frequency of mentions 
here. 

Similarly, respondents were asked to select three items that will be most 
affected within the next 3-5 years. The top two are the same as these 
mentioned for the current situation with development of appropriate teaching 
and learning strategies ranking third and accreditation studies and reports again 
ranking fourth. 

Or ganization for Measurement of Student Outcomes 

Over half of the institutions responding to the survey have designated a 
person to be responsible for directing or coordinating assessment for the 
purpose of assessing institutional effectiveness or assessing student outcomes, 
aud 37% have designated the same person to coordinate or direct both. Over 
half of the institutions have the assessment coordinator report directly to the 
president. 

Most often, respondents indicate that assessment was initiated because this 
complies with standard management practices. Curricular reform and the 
institutional governance structure prompted 25-30% of the instituti'^ns to initiate 
assessment, and 18% did so in response to actions or policies of an 
accreditation board or agency. The outcomes assessment systems of these 
institutions are most often developed by an office or department on campus as 
opposed to an administrative and/ or faculty committee, for example, and that 
is most often the office of institutional research. From a list of sixteen potential 
obstacles or impediments to implementing an outcomes assessment process, 
over half of the respondents select lack of personnel resources, and nearly as 
many indicate that lack of financial resources is a leading obstacle. Lack of an 
adequate database and lack of adequate measurement instruments are 
mentioned by 45% and 39% of the respondents, respectively. 

The effort of the histitutions in this survey is perceived more positively 
than the effectiveness of that effort. On a five point scale with 1 = 
low/negative and 5 = high/positive, effort ratings average 3.31 while 
effectiveness ratings average 3.03. This difference is statistically significant at 
the .000 level. 
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Analysis o f Current and Future Priorities 

Respondents expect the priority associated with the use of these Icinds of 
student outcomes to increase within the next 3-5 years, that is especially the 
case for student learning outcomes. 

38% indicate that thpre will be an increase in the priority associated with 
academic progress and employment outcomes 

54% indicate an increase for student learning outcomes, and 

36% indicate an increase in priority associated with student satisfaction 
outcomes. 

While current priority levels associated with the student outcomes 
measures are rather variable, there is general agreement that future priority 
levels will be high or the highest. That expectation for future priorities is 
fairly uniform, even for institutions that are currently not using the outcomes 
measures to assess institutional effectiveness. 

There is reason for frustration with the current situation and optimism 
with respect to what the situation might be within the next 3-5 years, 
lustration is evidenced in the finding that importance ratings for the use of 
the various types of outcomes measures are not related to current priority 
levels associated with their use; that is, average importance scores do not 
decrease with decreasing priority levels. Optimism, perhaps guarded, is 
evidenced m the fact that there is a relationship between future priority levels 
and importance scores. 

Use scores, on the other hand, are more highly related to current prioritits 
than to future priorities. This, again, is good news in that the priority expected 
to be assigned to the use of these outcomes over the next 3-5 years is basically 
at the high or highest levels. ' 

Rglationghips of Institutional Attributes to Using Student Outcomes Mpamirps 
to Assess Institutional Effectiveness — — 

Three attributes of institutions are hypotnesized to be related to the 
importance perceived for these outcomes and their use in assessing institutional 
effectiveness: enrollment size of institutions, regional accreditation affiliation, 
and geographical setting, i.e., urban, suburban, and so forth. 
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The findings do not support I'hese hypothesis. Only for mean importance 
scores on academic progress and employment outcomes is there a significant 
dilffierence by enrollment size, that is due only to the fact that the smallest 
institutions, those with an enrollment head count of 1,000 or less, ra' ? the 
importance of these measures lower than the larger institutions; the mean 
importance scores for the other enrollment categories are all equal. There are 
no significant differences in mean scores on the use index for any of the three 
sets of outcomes measuresi. 

Mean importance scores across the six regional accreditation associations 
are significantly different for only student satisfaction measures. Institu lions 
in the New England accreditation region have the lowest average importance 
score on student satisfaction outcomes, and schools in the other regions have 
about equal average importance ratings. It is interesting * note that schools 
in the Southern accreditation region do not rate the iiuportance of these 
measures significantly higher than do schools from some of the other regions. 

Mean use scores for both academic progress and employment outcomes 
and student satisfaction outcomes do differ significantly with respect to regional 
aCCTeditation membership; and for eoch of the outcomes measures, institutions 
in the Southern Association of Colleges and Schools do have the highest 
average scores on the use index. This is true for student learning outcomes as 
well, although the differences in mean scores are not statistically significant. 

Geographical setting bears no relationship to either the importance 
attached to using these outcomes to assess institutional effectiveness nor to the 
percentage of items in each tai of outcomes that is, on average, used by the 
institutions in the survey. 

Relationship of Having an Assessment Coordinator to Using Student Outcomes 
Measures to Assess Institutional Effectiveness 

Fifty-one percent of the institutions in the survey have a person who is 
responsible both for assessing student outcomes and institutional effectiveness, 
9% have designated an assessment coordinator for institutional effectiveness 
only, and 40% V :e neither or did not respond. 

Only for student satisfaction outcomes is there a significant difference in 
the average importance scores for institutions according to the assignment of 
responsibilities for assessment of student outcomes and /or institutional 
effectiveness. Use scores, however, do vary significantly with respect to 
coordinfj >n of assessment at the institution. Institutions that have designated 
respons ties for both student outcomes and institutional effectiveness average 
using more of the items in each set of outcomes measures. For student 
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learning outcomes, the average use score is about twice that of the other 
outcomes types. 



Current priority assignments have a low relationship to the coordinator 
role, but future priorities have no relationship. 

Institutions with no coordinator of assessment (or not responding to the 
question) are about twice as likely to experience low or very low satisfaction 
with the use of student outcomes for assessing institutional effectiveness as 
those with a designated coordinator(s) for both types of assessmem And, 
instthihons with a coordinator both of assessing institutional effectiveness and 
student outcomes are about twice as likely to experience very high satisfaction 
with the use of the assessments. Respondents at institutions that have assigned 
responsibilities for coordmating assessmem of both student outcomes and 
institutional effectiveness are more than twice as likely to rate their institutional 
assessment effort as "good" or "excellent" as those without a coordinator, and 
those without an assessment coordinator are almost three times more likely to 
give a poor or "very poor" rating to the institution's effort than respondents 
where both assessment responsibilities are assigned. 

Ratings of the effectiveness of that effort show similar patterns of 
differences although the magnitude of differences is not as great. 
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PART I 



AN OVERVIEW: USING STUDENT OUTCOMES 
MEASURES TO ASSESS INSTITUTIONAL EFFECTIVENESS 



Part I of this report covers the five sections of the questionnaire. 
Frequencies are reported in Appendix A for responses to each questionnaire 
item. 
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Questionnaire Development and Data Collection 

The primary components of the questionnaire were determined by a team 
sharing expertise and an interest in assessment, student success, and /or 
instituttonal effectiveness. Our goal was to maximize comprehensiveness with 
respect to reaching an understanding of what people in the two-year sector of 
higher education think about assessing institutional effectiveness, what they 
think about student outcomes assessment, and what they think about using 
student outcomes in assessing institutional effectiveness. We not only wanted 
to ascertain what these people think about the subject but also what is being 
done in two-year institutions to use student outcomes for assessing institutional 
effectiveness, how well that seems to be working, what uses are made of the 
assessment, and what is expected for the future. There are questions for 
people at institutions where no assessment is conducted and for those highly 
involved in the process. 

The questionnaire underwent many revisions ar.d reviews. One 
version~even longer-was p.'lot-tested by about 20 people. Another version was 
presented for discussion at \'he 1988 Project Cooperation Summer Conference. 
Many people contributed to many revisions of the questionnaire. 

The questionnaire was sent to the president of two-year institutions that 
are members of the American Association of Community and Junior Colleges 
(AACJC) in the fall of 1988. A reminder card and up to three letters were sent 
to institutions in an effort to gather as many respondents as possible. The final 
count is 675 unduplicated respondents, and that represents about 54% of the 
entire AACJC membership of two-year institutions. The reader is cautioned to 
keep in mind the fact that those responding to the survey may represent 
instituticns at the extremes-those that are more highly involved in the use of 
student outcomes measures for assessing institutior*al effectiveness or those least 
involved in this process. 

This response rate is rather good, particularly in light of the fact that the 
questionnaire is long and somewhat complicated. Quite a few people wrote 
or telephoned to ask for a copy of the questionnaire and/ or a copy of the one 
they had submitted. The message that we got over and over is that the survey 
instrument is comprehensive and that it serves as a useful stimulus or planning 
tool for institutions in the process of implementing new assessment 
procedures-in whole or in part~for the purpose of assessing institutional 
effectiveness and using student outcomes measures to do so. 

The questionnaire has five sections. The first three sections ask for 
information about three types or categories of student outcomes measures: (1) 
academic progress and employment outcomes, (2) student learning skills 
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outcomes, and (3) student satisfaction outcomes. The fourth section is aimed 
at ascertaining specific activities and functions for which the measures are used. 
And, the final section is used to gather information about how the assessment 
system is organized, administratively, and how it came into being. 

Additional data were collected from AACJC sources, for example, and 
added to the data set for each institution. 

Methodology and Reporting Procedures 

The report has two major parts. Part I provides a comprehensive 
overview m the form of frequency distributions for responses to each item in 
the survey. The actual frequencies are recorded in Appendix A along with a 
complete copy ;f the questionnaire. Unless specifically noted, the percentages 
reported in Part I are based on the total number of survey respondents (675) 
rather than the number of responses to an item. 

Part II provides more in-depth analyses of assessment practices. In that 
part of the report, findings are presented for analyses of whether institutional 
characteristics such as size, geographic setting, and regional accreditation 
assoaation affect assessment practices. In addition. Part II features an analysis 
of whether assessment practices vary between institutions that have a 
speafically designated coordinator or director of assessment and those that do 
not. 

Finally, the analysis focuses on comparisons of current assessment practices 
relative to future expectations for each type of outcome measure. In this 
section, a difference measure is computed that provides an indication of the 
magnitude of change between current and future priorities for each set of 
outcomes measures. 

Two indexes are computed and used extensively throughout the analyses 
in Part II. One index is based on respondent perceptions about the importance 
of using specific items included in each set of outcomes for assessing 
institutional effectiveness, and the other is compute ' ';om responses indicating 
that their institution actually uses the specific item(s) to assess institutional 
effectiveness. The first index is referred to as the importance score index and 
the second as the use index. Because these are used extensively, it is important 
that the reader understand how these are constructed and what they represent. 

Varying numbers of items are included in each of the three sets of 
outcomes measures examined in this survey. The set of academic progress and 
student outcomes measures includes 21 items, the set of student learning skills 
outcomes includes 16 items, and that for student satisfaction outcomes includes 
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11 items. Perhaps an example with the use index will be helpful taking two 
hypothetical institutions for this example, College Alpha dwd College Omega. 

The respondent for Alpha noted that Alpha uses 7 of the 21 items in the 
academic progress and employment outcomes, and that is 33% of the total. 
The respondent for College Omega noted that Omega uses 18 of the 21 items 
in this set of outcomes, and that is 86%. The use index for Alpha is 33% and 
for Omega is 86%. A use score was computed for each respondent in the 
survey. The mean or average for just these two colleges is 59.5%. 

The importance index was computed in the same way, but three 
alternatives were provided for the respondent to use in rating each item for 
each set of outcomes. Respondents were asked to indicate whether they 
perceive each item to be "not important," "important," or "very important" for 
assessing institutional effectiveness. These responses were coded 0 = "not 
important," 2 = "important," and 3 = "very important." For each respondent, 
the score on each item is computed by summing the coded responses. This 
sum is then divided by the to*-al number of points possible for each set of 
outcomes, eg. 63 is the maximum for academic progress and employment 
outcomes (21 items x 3 points = 63 points). Suppose that Alpha has responded 
to the 21 items so that the total number of importance points is 50; Alpha 
would then have an importance index score of 79%. 

SPSS-X was used for statistical analyses of the data. The entire two-year 
membership of AACJC was surveyed, therefore sampling techniques were not 
used. Sample statistics are reported, however, for the sake of convention and 
to give readers a basis of comparison. The statistics and analytical procedures 
presented in this report are simple and basic; most will be famihai to all 
readers of this report, and a brief review of the typical statistics is probably 
adequate. Statistics reported for tables of cross-tabulation (or contingency 
tables) are Pearson's product-moment correlation coefficient (r), the Chi-squaie 
value, the eta coefficient, and Cramer's V. 

The Chi-square statistic indicates whether there is any association between 
the variables. While that information is a necessary starting point, it is only 
that. It is also important to have some indication of the strength of association 
between variables. Cramei''s V provides such a measure for nominal level data. 
A desirable characteristic of Cramer's V is that it has upper and lower limits. 
Cramer's V eauals 0 when there is no relationship and it equals 1 when there 
is a perfect relationship. So, a significant Chi-square coefficient indicates that 
two variables are not independent, i.e., that there is a statistical association; and 
the value of Cramer's V indicates the level of relationship from very low tc 
very high. 
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The eta coefficient is more easily understood when compared to the r 
correlation coefficient. Eta is specifically for asymmetric relationships. While 
the product-moment correlation coefficient is not based on an assumption of 
linearity it does underestimate the level of association between two variables 
if the relationship is not linear. The eta coefficient will be greater than r for 
relationships that are not linear. Eta squared is inhiitively an analysis of 
variance measure. The higher the value, the greater the share of total variance 
accounted for across groups or categories on the nominal variable as opposed 
to within the groups. 

In analyses where means are reported, both r and eta are reported along 
with an F value. The F value is a ratio of variances such that the greater the 
value of F, the higher the probability that the means around which the 
variances are computed are not equal and that they are, in fact, significantly 
different. The F value is a ratio of the variance between the groups or 
categories relative to the variance within the groups. A large ratio means that 
the items within each group, i.e., respondents from institutions, are more alike 
on that particular dimension than they are like institutions in other groups. 

The F value can be larjge because only one or two means are very 
different from the others. It is necessary to look at other statistics such as a 
correlation coefficient or eta, to get a better idea of the relationship that exists 
between the variables. 
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Section I 



Academic Progress and Employment Outcomes 
Introduc tion 

Twenty-one items appear in the first section of student outcomes related 
to student academic progress and employment outcomes. Many of the items 
in this section are fairly traditional indicators of institutional effectiveness in the 
area of student retention, others have been used in recruiting students, faculty, 
financial support, and employment opportunities. Still other of these items are 
innovative and non-traditional measures of student outcomes relating to goal 
attainment and performance. 



A reliability analysis of these items reveals that the items have a level of 
internal consistency sufficient to be considered as an index (Nunnally: 1978; 
Richardson et al.: 1989). The average of all correlations among the items is .21 
with the minimum correlation being -.06 and the maximum being .79. The 
alpha coefficient of reliability is .85. 

Importance scale responses were coded 0 = "not important," 2 = "impor- 
tant" and 3 = "very important." If a respondent rated all 21 items as being 
very important for assessing institutional effectiveness, the sum would be 63 
points. Response scores were summed and divided by 63 to get an overall 
measure of the importance for each respondent. In Part II, this and other 
outcome importance and use scales will be further analyzed. For the present 
purpose of describing responses, one need note only that the range of 
importance scores is from a low of 16% to a high of 100%. A score of 16%, 
for example, means that this respondent generally rates these items as "not 
important," but gives an "important" rating to five items (5 x 2 = 10 points / 
63 = 16%), or a "very important" rating to two items (3x2=6) and an 
"important" ratii » to two others (2x2 = 4). In any case, only one respondent 
placed such low importance on the items and only 10% had an importance 
index score below 50%. Indeed, four respondents rate all items as "very 
important." The mean importance score is 70%. 

A use index was constructed in a similar, but much simpler, fashion. The 
product moment correlation between the importance and use scores is .30 and 
that is significant at the .000 level. The percentage of items used is the use 
index, and it ranges from 0.00% to 100%. The modal response is "none": 91 
of the institutions currently use none of these items to assess institutional 
effectiveness. Note that the institutions could collect data on some of the items 
but not use the information in assessing institutional effectiveness, per se. The 
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mean use score is 37%. Half of the institutions responding to the survey use 
at least 33% of the items-that is, seven or more items. About 90% of the 
responding institutions use up to 14 of these items, and eight respondents 
indicate their institution is already using all of these measures for assessing 
institutional effectiveness. That is rather phenomenal. 

Part A: I mportance Ratines and Use of Academic Progress and Employment 
Outcomes Measures 

From the list of 21 measures of academic progress and employment 
outcomes, the measure most widely used by respondents to this survey in 
assessing institutional effectiveness is the number of students who graduate 
with an associate degree. This is one of the more "traditional" indicators of 
institutional effectiveness, and 73% of the respondents indicate that it is used 
at their institution. While that is the most used measure of academic progress 
and employment outcomes, the highest importance rating goes to employer 
satisfaction with graduates: 88% of the respondents rate this outcome as very 
important and 9% rate it as important. A closely related measure received 
nearly as many "very important" ratings: 87% rate the measure of employer 
satisfaction with job training/skills enhancements as very important and 10% 
rate it important. These two employer assessments of student outcomes are 
each used by about 60% of the responding institutions to assess their 
effectiveness. 

Perhaps the all-around most favored measure in the academic progress and 
employment outcomes list is item #14: "the percentage of graduates who find 
employment in their major/field of study." This item is the second most 
frequently used measure of institutional effectiveness with 70% of respondents 
saying their institution uses it. In addition, the importance rating is very near 
the top rating with 84% saying it is very important and 12% saying it is an 
important indicator of institutional effectiveness. 

Two other measures are currently used by over half of the respondents: 
"the percentage of graduates who complete their intended program or degree 
at your institution" is used by 59% of these institutions and just over three- 
quarters of the respondents rate this measure as a very important indicator of 
institutional effectiveness. The percentage of graduates who find employment 
NOT related to their major/field of study is used by 54% of these institutions; 
51% of the respondents rate it as a very important indicator; and another 40% 
rate it as important. 

To respond that any one of these measures is "not important" is more 
difficult than to rate its importance as "low." Knowing that to be the case, 
however, the NOT important response was placed on the questionnaire in order 
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to maximize the scale differences and increase the discrimination between 
moderate importance and the lower end of the scale. Recognizing that 
respondents might feel reluctant to say that the items are not important, it is 
worth noting that four of the 21 items have at least one quarter of the 
institutions giving this response, and another three items have 23% NOT 
important responses. The leading "not important" items are as follows: 

- the percentage of transfer students who do not receive a bachelor's degree 
within a specified time: 40% not important 

- the percentage of students in an associate degree program that earn the 
degree in three years: 33% not important 

- annual total income of graduates: 28% not important 
(NOTE: This item has the lowest rate of use.) 

- of those enrolled annually, the percentage of students who intend to 
transfer without earning the associate degree: 24% not important 

- grade point average of f.tudents who transfer without a degree relative to 
native students of the four-year institution: 23% not important 

- grade point average of students who transfer before completing a degree 
program: 23% not important 

Part B: Type of Use 

Respondents were next asked to indicate the general use made of this type 
of student outcomes. Only 12% of the respondents say that academic progress 
and employment outcomes are not assessed at their institution; 25% assess 
these outcomes but the measures are not used to assess institutional effective- 
ness; and 61% say that these outcomes are both measured and used to assess 
institutional effectiveness. 



Part C: Most Important Outcome Measure Used Now and in Future 

Part C asks respondents to select an item from the list that is now most 
important and then select one that will be most important in 3-5 years. Several 
respondents revealed frustration with this question because they found it too 
difficult to identify a single item as the most important. Some would write 
"All of them" and others would list multiple items. The only feasible solution 
was to consider these multiple responses as missing 'lata. Respondents that 
indicated non-use on the previous item were instructed to skip this question. 
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That set of "inappropriate" respondents accounts for a little over one-half of the 
no response and missing data cases. 

Respondents tended to single out item #14 (percentage of students who 
find employment in their major/field of study) as the most important measure 
currently used to assess institutional effectiveness. Over one-quarter of the 
institutions represented in the survey (27%) selected this item, and 15% selected 
item #7: "the percentage of students who complete their intended program, or 
degree at your institution." Recall that item #14 had the second highest use 
rating and the third highest number of respondents ratine it as "verv 
important." o j 

Items #1 and #17 tied for third place in frequency of mentions as most 
important academic performance and employment outcomes items for assessing 
institutional effectiveness. This is interesting when compared to the ratings of 
individual items. Item #1 liad the highest use rating and item #17 the highest 
importance rating, yet neither item netted responses on this question to place 
it higher than third in importance among outcomes measures of this type 
currently used for assessing institutional effectiveness. 

With respect to expectations about the near future, the items most often 
mentioned as being most important are the same as those currently used with 
one exception. Only 3% of respondents expect item #1 (students graduating 
with an associate degree) to be the most important measure in assessing 
institutional effectiveness in the future. That places it in fifth position relative 
to other items. Item #16 (employer satisfaction) enters the list in third position 
as the most important measure of the future. Heading the list for future 
importance is item #7 (the percentage of students who fulfill their goal at the 
institution) and that ties at 21% of the institutions with item #14 (the 
percentage of graduates who find employment in their major/field of study). 
The other academic progress item that will be important io assessing 
institutional effectiveness is employer satisfaction with graduates-the item that 
got the highest number of "very important" responses in Part A. 

Parts D a nd E; Current and Future Priorities 

Parts D and E again focus attention on current practices and future 
expectations. Respondents were first asked to rate the priority level currently 
associated with these measures relative to other outcomes actually used to 
assess institutional effectiveness. Again, the 80 institutions that do not measure 
these outcomes were instructed to skip this current priority item and indicate 
only the priority expected to be associated with these measures within the next 
3-5 years. 
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At present, 13% of the respondents rate academic progress and employ- 
ment outcome? as outcomes that are given far higher priority than any other 
outcomes to assess institutional effectiveness. The majority, 54%, respond that 
these items are given higher priority than other outcomes; 16% r'^spond that 
these items are just a part of the criteria used for assessing institutional 
effectiveness but they are not high priority items; and just 1% respond that of 
the items used to assess institutional effectiveness, these are least important and 
receive no priority whatsoever. 

Future expectations are for the priority given these measures to increase 
within the next 3-5 yearj. Responses indicate that institutions giving the 
highest priority to these measures vvill double the current rate, and a slightly 
higher rate expect to give high priority to these measures. These future 
expectations to increase priority, of course, mean a shift from lower and no 
priority categories. Furttier analysis of the current and future responses for 
these and other outcomes will be presented in Section n of this report. 

Part F: Satisfaction With Use of Outcomes Measures 

The final question in the section on academic progress and employment 
outcomes asks respondents to rate the level of satisfaction at the college with 
the use of the outcomes measures for assessing institutional effectiveness. The 
mean sco^^ on this five-point scale is 3.08, and that shows medium satisfaction. 
"Medium satisfaction (scale value = 3) is the mode for responses, and 28% 
responded that satisfaction levels are high or very high while 21% responded 
that satisfaction is low or very low. 

Summary 

In general, these outcomes are found to have a rather high use rate, to be 
regarded as important for assessing institutional effectiveness, and to meet with 
medium to high satisfaction in use. The future should bring even higher and 
more intensive use rates, based on comparisons of priorities now and in the 
near future. 
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Section II 



Student Learning Outcomes 
Introduction 

The second section of the questionnaire is organized exactly the same as 
the previous except for the addition of a part asking for more information on 
the way(s) the skills are measured. A total of sixteen learning outcomes are 
included in the section covering measures of student learning skills. 

These learning outcomes are grouped under three headings: foundation 
skills, process skills, and competency in general education or field of study. 
Foundation skills were expected to be most often used and process skills least 
used. Further, to the extent that these skills are assessed, the expectation is 
that this is largely an "on-entry" assessment, that few institutions are involved 
in any type of interim or exit assessment, and that few institutions use the 
assessment data that are gathered for purposes relating to assessing institutional 
effectiveness. Before turning to the findings regarding the measures of student 
learning outcomes, a brief description of the items in this section should prove 
useful. 

The reliability analysis of these sixteen items indicates even higher internal 
consistency among the items. The alpha coefficient is .8^, and the average 
inter-item correlation is .34. Nunnally (1978: 350) says that an average inter- 
correlation over .30 is exceptional. The range of correlation here is from .09 to 
.72. 

The average score on the importance index is 83%. The distribution is 
highly skewed so that very few respondents gave low importance ratings and 
many gave high ratings. Only 3% of respondents participating in the survey 
gave importance ratings of 50% or less, and the median importance score is 
85%. Finally, 38 respondents rated all sixteen items as "very important," 
yielding an importance score of 100%. 

Relatively few institutions use these measures for assessing iustitutional 
effectiveness. Nearly half of the respondents (46%) indicate that none of these 
items are used to assess institutional effectiveness, and the average use score 
is only 20%~an average of just over 3 of the 16 items. The median is only 1 
item used for this purpose, and over 85% of the respondents use no more than 
half of these items to assess institutional effectiveness. The expectation that 
these outcomes are not widely used is overwhelmingly fulfilled. 
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Part A; Importance Ratings and Use of Measures 

As the previous overview revealed, the importance attached to these 
measures is high and the use rate is low. Indeed, the correlation between the 
importance and use scales is only .18 (significant at the .000 level). 

The skill most frequently used in assessment of institutional effectiveness 
is a foundation skill: writing skills. Just over 40% of the respondents (43%) 
use measures of writing skills and that use rate is followed by measures of 
reading (39%), arithmetic skUls (37%), and mathematics skills (29%)-all in the 
foundation skills category. The more general and comprehensive skill measures 
rank fifth and sixth in frequency of use with field of study competency 
measures used by 31%, and general education competency measures used by 
26% of respondents. Process skills stand out for the low usage of these skills 
in assessing institutional effectiveness. The low use rates for process skills is 
in sharp contrast to the high importance ratings given to the measurement of 
these skills. 

The least important indicators of institutional effectiveness are outcomes 
measures based on advanced mathematics skills. This skill area got more "not 
important" responses (22%) than "very important" (19%). Only three other skills 
were rated as very important indicators of institutional effectiveness by less 
than half of the respondents, and those were each very near the 50% level. 



Part B; Type of Use 

Further confirmation of relatively low use of these student learning skills 
measures is found in responses to these items in Part B. 

o 23% of the institutions in the response set do not assess these skills. 

o 43% assess this type of skill, but the assessment is not used to assess 
institutional effectiveness. 

o 35% of the institution both assess this type of student learning skills 
and use the assessment in assessing institutional effectiveness. 



Self-understanding 
Social Responsibility 
Computer Literacy 



48% 
46% 
44% 
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Part C; Measurement of Student Learning Skills 



Type of Assessment 

For this section on student learning skills, respondents were asked to 
provide more detailed information about the assessment methods used at their 
institution. If the skill is measured but not used in assessing institutional 
effectiveness, the respondent gives one response; if the skill is both measured 
and used in assessing institutional effectiveness, the respondent gives another 
response. If that skill is not assessed at all, the respondent leaves the space 
blank. The greatest number of institutions are involved in some type of 
assessment of reading skills and the smallest number are engaged in assessment 
of life-long/self-directed learning skills. The average number of institutions 
engaged in some type of assessment of these sixteen learning skills is 212. The 
activity level seems to be much higher for the foundation skills, and an average 
of 326 institutions conduct som.e type of assessment in these eight skill areas. 
The number of institutions that assess general education competency or fields 
of studv competency averages to 178. Process skills areas are assessed by 
relatively few institutions-a low of 46 to a high of 120. That is a range of 
from only 7% to 18% of the institutions responding to the survey that are 
involved with any sort of process skills assessment. The average number of 
institutions assessing the process skills included on the questionnaire is only 
72-about 11% of the survey institutions. 

As previously mentioned, the expectation was that entry level assessment 
is more typical than interim or exit assessment, and that a comparison of an 
entry wim later/exit assessment is even less typical of institutions of higher 
education. That expectation is somewhat substantiated by the findings in this 
part. The number of skill areas for which more institutions conduct some type 
of exit assessment is greater than that for entry assessment: exit assessments 
are conducted at more institutions than entry assessments for 10 of the 16 skill 
areas. Over the 16 skill areas, however, entry assessments are mentioned by 
an average of 168 institutions which is an average of 25% of the institutions 
participating in the survey. This is a higher rate than for exit or comparison 
assessments. The average of 104 institutions--15% of the total-that conduct exit 
assessments of these student learning skiMs exceeds entry/exit comparisons 
assessments for which the average number of institutions is only 40-that is, 6% 
of the total. Entry level assessments are clearly conducted by far more 
institutions than exit only or entry/exit comparison assessments. Exit 
assessments exceed entry level assessments for the less often used process skills 
and more general competency measures. 

Recall that more institutions use writing skills for assessing institutional 
effectiveness than any of these other student learning skill. The leading 
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assessment activity according to responses on this part of the questionnaire is 
entry level assessment of reading skills. Over three-quarters (78%) of the 
institutions are involved in entry assessment of reading skills, but only 14% 
indicate that the results are used in assessing institutional effectiveness. 
Slightly fewer-73%~of the institutions carry out entry level assessment of 
wnting skills, and this is followed closely by entry level assessment of 
mathematics skills and arithmetic skills (70% each). 

Entry assessments of writing, reading, arithmetic and mathematics skills are 
about equally used for assessing institutional effectiveness-13% - 14% use rates 
are reported. This is much lower than the 57% to 63% rate of assessment 
reported by these institutions. 

More institutions conduct entry-exit comparison assessments on reading 
skills than any of the other student learning skills, but this is nearly matched 
by the number involved in this type of assessment of writing skills (128 and 
122 institutions, respectively). Again, mathematics and arithmetic skills are the 
two areas also assessed by a larger number of institutions-99 and 98, 
respectively conduct entry and exit assessments in such a way that comparisons 
are possible. 

The differences in assessment type used for these student learning skills 
are more apparent when the responses are examined somewhat differentiy. 
SPSS-X has a multiple response analysis program that provides the opportunity 
to gain more information on how respondents answered items for which more 
than one response is logically possible and is coded in the data set. Type of 
assessment is an example of a multiple respoi\se variable. Logically, an 
institution may use entry and exit assessments of a skill, but results of the 
particular assessments may not be amenable to comparison. Or, the institution 
may use entry assessment for placement purposes and also conduct assessments 
that are designed for entry-exit comparisons. Thus, respondents could check 
none of the assessment types or all three for any given learning skill (see 
page 82). 

When assessment type is computed as a percentage of the responses, the 
average number of exit assessments exceeds entry assessments slightiy-46% vs. 
43%, and the average share of entry-exit comparison responses is 11% for these 
16 learning skills. 

Type of Instrument(s) Used 

Respondents were next asked to indicate the type of instrument(s) used to 
measure these skill areas. Choices include national or district instruments, state 
inslTuments, campus instruments or classroom instruments. Over the 16 
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student learning areas, an average of 76 institutions can be expected to use 
classroom assessments of the skills. Instruments developed for use campus- 
wide have the next highest use rate, and the average number of institutions 
using campus instruments to assess these student learning skills is 66. District 
or national instruments are used by an average of 63 institutions. State 
instruments are rather uncommon: on average, only 22 institutions would be 
expected to use state instruments to assess any one of these 16 learning skills. 

The final item of information requested in this section is the particular 
instrument used in assessing the various learning skills. Considerable 
variability exists in instruments used for assessing these skill areas. Codes 
were established for the 25 most frequently mentioned instruments. The reader 
should be aware of the fact that respondents would often write "ACT" or "SAT" 
or "CEEB" when, in fact, the instrument being used is ASSET or COMP, for 
example. Nonetheless, the actual instrument mentioned is coded and reported 
here. Only the seven specific instruments most often mentioned are reported 
in Appendix A, pages 83-84. In addition, mentions of either the Texas, 
Georgia, or Florida assessments are reported under "State Tests" despite the fact 
that respondents were instructed to name only national instruments. Finally, 
an "AH Others" category includes those miscellaneous responses that were 
originally coded into the "Others" category- plus those low frequency responses 
that were originally assigned codes. 

ASSET dominates assessment in all foundation skills areas. About one- 
third of all two-year institutions use ASSET, so the high rate of use reported 
here should not be too surprising. Specific test instruments do stand out for 
use in specific skill areas. For exaniple, 21% (74) of the institutions that report 
an instrument for assessing reading skills name the Nelson-Denny test. Myers- 
Briggs is the instrument used to assess students' self-understanding skills in 
22% of the 18 institutions that name assessment instruments for that skill. 
Just over 40% of the institutions naming instruments for assessing both 
aesthetic appreciation (14) ai.d social responsibility (17) skills use the COMP. 
And, nearly that large a share, 38%, of the institutions mentioning instruments 
for assessing field of study competency name the NLN. 

Part D; M ost Important Outcome Measure Used Now and In Future 

Respondents seemed to have even more trouble responding to these items 
than they did for the same question on the previous set of outcomes measures. 
Again, the difficulty appears to have resulted from the instruction to identify 
only one item from the list of 16. A quarter of the respondents (26%) failed 
to respond at all to this question or failed to identify only one item, and 
another 23% were instructed not to respond since this type of outcome is not 
assessed at the institution. Only about half of the institutions in the survey 
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have responses reported to this item. Reading skills assessments were most 
often mentioned (18%) as being the most important of these 16 skill areas 
currently used to assess institutional effectiveness, and this is followed by 
assessments of field of study competency (13%), writing skills (11%) and 
general education competency (6%). These four items are among the top six 
in importance ratings and use rates reported in Part A. 

Responses regarding expectations for the future appear to have been less 
difficult since more responses are recorded here. These responses reflect the 
general trends evident in the literature that would move assessment more 
toward general/comprehensive learning and thinking skills and less toward 
individual skills in the foundation skills area. Specifically, assessment of 
general education competencies is most frequently named as the student 
learning outcome that will be most important within the near future (15%), and 
measures of competency in the student's major or field of study is the second 
most frequently mentioned (12%). Reading skills and writing skills come in 
third and fifth positions with 10% and 1%, respectively, of the respondents 
mentioning one as most important. Assessment of critical thinking skills is the 
fourth most frequently mentioned (8%) student learning outcome to be used in 
assessing institutional effectiveness in the next 3-5 years. This process skill is 
chosen as the sinjgle most important student learning outcome for assessing 
institutional effectiveness in the future by 51 responding institutions as opposed 
to only 3 who give it that singular importance in today's assessment picture. 

Parts E and F; Current and Future Priorities 

Again, the comparison of expectations for future and current pricrities 
associated with student learning outcomes shows that considerable change is 
anticipated in the role this type of outcome measure will assume in the 
assessment of institutional effectiveness. 

During the next 3-5 years, the number of institutions that give high 
priority or far higher priority to student learning outcomes will nearly double 
over current rates (48% vs. 83%), and almost three times more institutions will 
be giving far higher priority to these measures than is currently the case (29% 
vs. 10%). ' 

Part G; Sa tisfaction with Use of Outcomes Measures 

Respondents were asked to rate the overall satisfaction at their college with 
how these measures are used for assessing institutional effectiveness. The 
mean satisfaction score is only 2.65 indicating that on average, respondents 
perceive lower than medium satisfaction with how these measures are used on 
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their campus. Responses on the lower side of medium satisfaction total 39% 
while those on the higher side total 16%. 

Summary 

In general, there is considerable divergence between the role respondents 
seem to find appropriate for measures of student learning outcomes and the 
role currently given these measures. To the extent that student learning skills 
are measured, the typical pattern is for entry only or exit only measurement; 
more often, entry only. Further, assessment is concentrated in basic skills 
rather than process or general competency areas. 

Respondents anticipate considerable change in the near future so that these 
types of measures will experience a sharp increase in priorities associated with 
their use for assessing institutional effectiveness, and the assessment focus will 
shift away from basic skills and move toward general education/ field of study 
competencies. 




Section III 



Student Satisfaction Outcomes Measures 
Inti-oduction 

The final section specifically related to attitudes and opinions about student 
outcomes and their use in assessing institutional e/fectiveness covers measures 
of student satisfaction. Only 11 items appear in this section, and these were 
again chosen to be comprehensive in covering many facets of a student's 
experiences on campus from the classroom, residence halls, services, and the 
campus in general. 

The alpha coefficient from the reliability analysis of this set of items is .87 
and the average inter-item correlation is .38. These figures indicate that these 
items have a high level of internal consistency. The minimum inter-item 
correlation is .16 and the maximum is .60, and this set of items has consider- 
ably lower variation than the previous sets. 

The average score on the importance index is 85% and the median is also 
around 85%, but the modal score is 100%. 

All of these student satisfaction measures were rated as very important 
(100% score) for assessing institutional effectiveness by 16% of the respondents 
which is 108 institutions. Further evidence of the relatively high level of 
importance attached to these items is found in the fact that all but 1% of the 
respondents awarded at least 50% of the importance points to these items. 
That is to say that over 99% of the respondents would award at least half of 
the importance points to these items. 

The number of institutions that use none of these items to assess 
institiitional effectiveness is rather high, compared to the importance attached 
to the measures for this purpose: one-third (33%) of the institutions use none. 
Half of the institutions use no more than four cf these measures, while 42% of 
the institutions use at least half of the measures in their assessment of 
institutional effectiveness. O'-.erall, use is oi; the lower side among these 
institutions. 

Part A; Importance Ratings and Use of Measures 

The overview shows that the importance of these measures is rated quite 
high but use is relatively low. The correlation between importance and use 
scores is .28. 
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The satisfaction measure receiving the largest number of "very important" 
responses relates to faculty: quality, attitude toward students, availability, and 
so forth. This item is rated "very important" a. 83% of the institutions in the 
survey. The second highest number of "very important" ratings is for measures 
of satisfaction with the curriculum. Respondents at 15% of the institutions 
believe it is very important to use information about student satisfaction with 
*^Pf5u.i?^ curriculum such as the variety of courses, qual'ty, content, 
availability and class size. Another item felt to be very important is student 
satisfaction with academic support services: 72% of the respondents rate this 
as very important, and nearly that number-70%--rate satisfaction with the 
quality, content, and availability of academic advising as very important. Two 
other items in this set are rated very important by over half of the survey 
participants: 61% rate satisfaction with career planning and skills enhancement 
at the highest importance level and 50% give that rating to the importance of 
satisfaction with official procedures such as admissions, registration, change of 
major, and so forth, ^ 

A relatively low rate of "very important" responses on these items is not 
accompanied by a high rate of "not important" responses. On average, less 
than 3% of the "ispondents gave that importance rating. The item with the 
most negative importance rating is satisfaction with facilities such as oor- 
mitories, classrooms, and campus grounds generally. A total of 55 respondents 
(8%) reject the importance of this item for assessing institutional effrctiveness. 
Given the fact that few of these colleges are residential campuses, this is 
perhaps a surprisingly low number of not important responses. It is, of course, 
the case that other significant facilities such as classrooms were specifically 
mentioned on the item. 

Four of these satisfaction measures are used by over half of the institutions 
to assess effectiveness. These are satisfaction with faculty (59%), satisfaction 
with academic support services (52%), satisfaction with curriculum offerings, 
and satisfaction with courses in the student's major area of study (50%). The 
least used satisfaction measure is that related to campus conditions in general 
(27%). ^ 

Part B; Type of Use 

About 18% of the institutions do not assess student satisfaction, 26% assess 
shident satisfaction but do not use the information as input to their assessment 
of institutional effectiveness, and 55% of the institutions use student satisfaction 
measures to assess institutional effectiveness. 
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Part C; Most Important Outcome Measure Used Now and In Future 

Only three of the satisfaction measures stand out with respect to the 
number of institutions that identify them as the most important of the 
satisfaction measures currently used in assessing institutional effectiveness. 

Satisfaction with curriculum = 28% 
Satisfaction with faculty = 22% 

Satisfaction with courses in major or field of study = 13% 

These same three are expected to be the most important satisfaction 
measures used to assess institutional effectiveness within the next 3-5 years. 

Satisfaction with curriculum = 28% 

Satisfaction with courses in major or field of study = 15% 

Satisfaction with faculty = 13% 

Hiese represent the core of the institution's instructional efforts, and it 
would be difficult to argue with the current and future importance of these 
measures in assessing institutional effectiveness. It is somewhat surprising that 
measures such as satisfaction with procedures, services, and general conditions 
are so little valued. Other items that show change in importance in the future 
are satisfaction with career planning/skills enhancement and satisfaction with 
acadenuc support services. 

Parts D and E: Current and Future Priorities 

Responses on these parts of the surve^' indicate a less static future for 
these satisfaction measures than the results of the preceding part would 
suggest. Yet, the change anticipated with respect to priorities to be placed on 
these satisfaction measures is minor, relative to previous outcomes measures 
examined in the survey. 

The change in the "far higher priority" responses from the current use to 
future use is only an increase of about 50%, and there is only about a 35-40% 
increase at the next highest level. While the priority associated with these 
measures is clearly expected to increase, the change is not expected to be great 
relative to other types of outcomes measures. 

Part F: Satisfaction With the Use of the Outcomes Measures 

The average satisfaction with the use of student outcomes measures to 
assess institutional effectiveness is 3.07. That is about the same as satisfaction 
with the use of academic progress and employment outcomes, and these are 
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both considerably higher than the average satisfaction with how student 
learning measures are used. This average means that the typical institution is 
expected to feel medium satisfaction with the use of these outcomes for 
assessing institutional effectiveness. Overall, around 23% of the institutions in 
the survey rate satisfaction with use as low or very low, 37% rate satisfaction 
as medium, and 31% enjoy high or very high satisfaction with the current use 
of these measures for assessing institutional effectiveness. 

Summary 

Once again, the findings indicate that these outcomes measures are 
perceived to be important, but at present use is not as great as the importance 
ratings would suggest. 

While respondents expect their institutions to raise the priority given these 
measures within the next 3-5 years, the change expected for studen' satisfaction 
outcomes measures would appear to be considerably less than for measures of 
student learning skills outcomes. 

Summary for Sections I - III 

Student satisfaction outcomes and student learning outcomes are perceived 
to be about equally important according to their importance index scores, and 
their importance is perceived to be considerably higher than that of academic 
progress and employment outcomes, overall. The means for each importance 
indexes are statistically significantly different at the 98% level or higher. 

When use scores are viewed, however, academic progress and employment 
outcomes and student satisfaction outcomes are closer, on average, than student 
learning outcomes. The mean importance score for student learning outcomes 
is 83% while the mean use score is only 20%. T-tests on means for the use 
index scores shows that the means are statistically significantly different. The 
two-tailed probability for each of these means tests is .000. 

No matter which outcome type-respondents expect usage of these 
measures to increase in the future and they expect this use to be given higher 
priority within the near future. The greatest changes are expected for use of 
student learning skills outcomes, and use of student satisfaction outcomes is 
expected to be most stable. 

Sa'isfaction with how academic progress and employment outcomes and 
student satisfaction outcomes are currently used to assess institutional 
effectiveness is about equal, and the difference in the mean scores on these 
scales is not statistically significant. While respondents rate use of these 



34 



ERIC 



38 



outcomes with medium satisfaction, the average satisfaction with the use of 
student learning outcomes is statistically significantly lower. 

Since less use of student learning outcomes was anticipated, additional 
information was collected to better understand the ways these are measured. 
As one would expect, foundation skills are most often measured and used in 
assessing institutional effectiveness, and process skills least so. Further, to the 
extent that student learning skills are measured and/or used to assess 
institutional effectiveness, entry level is most commonly followed by exit 
assessment. Very little assessment amenable to a "value-added" interpretation 
is conducted. 
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Section IV 



Uses of Outcomes Assessment Infomation 

In this section of the qutjtionnaire, respondents were presented a list of 
25 activities and functions tivought to be typical of most institutions of higher 
education, particularly those in the two-year sector. 

The three sections on outcomes measures are based on the assumption that 
some items are not measured at all; some are measured but this measurement 
or assessment is not an input factor into the assessment of institutional 
effectiveness; and some are not only measured but also serve as inputs into the 
assessment of institutional effectiveness. 

This section focuses on the feedback effect that may be assumed for 
assessments of institutional effectiveness. If student outcomes are measured 
and used to assess institutional effectiveness, then this assessment is perhaps 
fed back into the process of policy making and planning so that typical 
activities and functions of the institution are affected. 

Part A; Activities and Functions Affected by Student Outcomes Measures 

Respondents were asked to place a check if any of the three types of 
outcomes has been used with respect to each activity or function. They could 
mark all three, any two, any one, or none. 

Academic progress and employment outcomes seem to be most used c': 
the three types of outcomes. Across all 25 activities, an average of 45% of the 
institutions use measures of academic progress and employment outcomes, 34% 
use measures of student learning skills outcomes, and 33% use measures of 
student satisfaction outcomes-a total average of 37% using these outcomes ;o 
affect these activities and functions. 

The number of affirmative responses across the 25 activities was counted 
for each institution and the percentage of institutions using student outcomes 
to impact from 0 to all 25 activities computed. The results are presented on 

gages 94-95 in Appendix A just after the responses to Part A. As this table of 
■equency distributions shows, the average number of activities for which 
academic progress and employment outcomes are used is 11.3. An average 
of 8.7 activities and functions are affected by student satisfaction outcomes, and 
8.4 activities and functions are, on average, affected by student learning 
outcomes measures. The mean for academic progress and employment 
outcomes is significantly different from that of both student learning and 
student satisfaction outcomes measures, but means for student learning and 
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student satisfaction outcomes are not significantly different (t=-1.5, two-tailed 
probability=.14). 

Accreditation studies and reports are most often impacted by the use of 
these three outcomes measures. About three-quarters of the institutions use 
measures of academic progress and employment outcomes for accreditation 
studies and reports, over half (52%) use student learning skills outcomes, and 
58% use student satisfaction outcomes measures for this purpose (See 
responses to Part A of this section in Appendix A). 

Again the number of affirmative responses across the 25 activities was 
counted for each institution, and the percentage of institutions using each set 
of student outcomes to affect various numbers of activities was computed. The 
frequency distribution table on pages 95-96 of Appendix A shows accreditation 
studies to be the activity most affected by all three outcomes: 44% of the 
institutions use all three types of student outcomes in accreditation studies, 18% 
use two of the three types of outcomes, 16% use only one, and 21 % use none 
of these outcomes for accreditation studies. 

The second highest aveiage use-nearly 60%-for the three outcomes is 
found to be for institutional planning, in general. About 42% of the insti^^^tions 
use all three outcomes lo further their institutional planning efforts while 23% 
indicate that they use none (see page 96). That is followed in frequency of 
mentions as an impact on curriculum development: an average of 52% of the 
institutions use one of these outcomes in curriculum development. Referring 
again to the frequency table on page 95, one sees that 24% of the institutions 
actually use all three outcomes to affect curriculum development. And, only 
18% indicate that none of the measures is used to affect curriculum develop- 
ment; that is, the lowest rate of non-impact of all 25 activities. 

At the other end, these outcomes are least used for providing faculty 
incentives. The average is only 15% for the three outcomes together, and 
student learning outcomes are least used-only 10%-in relation to faculty 
incentives. The frequencv table on page 95 shows that nearly three-quarters 
of the respondents (72%; indicate that none of these outcomes is used to 
impact faculty incentives. This, in conjunction with the other little-impacted 
activities and functions, describes a situation worthy of note. The other 
activities and functions that are affected by these outcomes at relatively few 
institutions are development of alternative instructional modes (24%), creation 
of new positions (24%), economic and human resource development (24%), and 
funding purposes (26%). Over half of the institutions indicate that none of the 
outcomes is used to affect these four activities and functions. With the 
exception of the alternative instructional modes, these activities and functions 
represent potentially punitive uses or application of outcomes to which faculty, 
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in particular, object. This is not to say that using these outcomes for the 
purposes listed here is necessarily negative. Indeed, some institutioas do have 
funding, staffing, and personnel promotional dt cisio..^ tied to these outcomes 
measures, and many people believe this will increasingly be the case in the 
future. 

Part B; Activities Curre ntly Most Affected bv Measures of Student Outcomes 

Respondents were asked to select three items from the list of 25 that 
represent activities and functions currently most affected by the use of student 
outcomes measures and then to indicate which set(s) of outcomes most affectod 
the activity or function. 

Responses will be presented in two ways. First, the activities and 
functions are presented according to the sequence in which they were named. 
Next the most frequently named activities and functions will be presented, 
regardless of the order of mention. 

As seen in the table for Part B of this section in Appendix A, curriculum 
development is the most frequentiy (28%) mentioned function of those listed 
first, course offerings leads (9%) among those listed second, and institutional 
planning is the most often named function (9%) in the third selection. 
Curriculum development is certainly the overwhelming selection as the activity 
that stands out to these respondents for having been most affected by the use 
of student outcomes measures. 

The student outcomes measures most often mentioned in conjunction with 
the items-whether named first, second, or third-are academic progress and 
employment outcomes. 

The second mode of analyzing these data uses the multiple response 
program in SPSS-X, and these results are presented on page 93 in Appendix A. 
The three items named as being most affected are combined as a single 
variable. The outcomes named as impacting the variables are recoded into 
seven patterns of use: 

1 academic progress and employment outcomes used alone 

2. student learning skills outcomes used alone 

3. student satisfaction outcomes used alone 

4. academic progress and employment outcomes used along with student 
learning skills outcomes 

5. academic progress and employment outcomes used along with student 
satisfaction outcomes 
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6. student learning skills outcomes used along with student satisfaction 
outcon\es 

7. all thr&«; types of outcomes are used together. 

Respondents could give one of these impact patterns along with eacU of 
the three activities or functions impacted by the outcome(s). 

A total of 1562 item responses were given. Practically all of the 
respondents to this question named three items. From the list of 25 items, the 
11 most frequently mentioned were selected for analyzing in conjunction with 
the outcomes named as impacting the activities or functions. From this list, 
curriculum development is the activity most often mentioned: it was 
mentioned 250 times which is 16% of the total mentions and 48% of the 
institutions in the survey. Institutional planning in general is next in frequency 
of responses with 28% of the institutions mentioning it for 9% of the total 
number of mentions. Course offerings is next in impacted activities: 130 of 
the total number of responses v ere "course offerings" and that represents 
responses of 25% of the institutions. Finally, about 5% of the mentions were 
of accreditation studies, and that is 15% of the institutions. The "all others" 
group of responses accounts for 24% of all responses, and nearly three-quarters 
of the respondents (72%) mentioned at least one of these 14 other activities or 
functions among the three items named. 

When the item responses are investigated along with the 7 outcomes 
patterns listed above, the number of responses counted in the multiple response 
cross tabulation is 3537. Each pairing or crossing of outcomes pattern with 
activity counts as two responses (SPSS-X). Those data are presented in a table 
on page 101 of Appendix A. The pattern of outcomes most often used at 
present is 1) academic progress and employment outcomes only (30%), and that 
is followed by use of 7) all three outcomes together (25%). Somewhat 
surprising is the fact that 14% of the respondents indicate that student learning 
skills is the single outcome currently most affecting the selected activities. This 
is surprising in light of the finding that this set of outcomes measures has the 
lowest use rate. Perhaps this means that student learning outcomes measures 
are especially effective. 

Much of the impact of using measures of student learning skills outcomes 
is in course placement. Over one-third of the respondents (39%) mentioning 
course placement as an activity that has been highly affected by the use of 
outcomes measures name student learning skills as the sole outcome impacting 
this activity. In fact, over half (59%) of the respondents who name course 
placement as one of the three most affected activities indicate that measures of 
student learning skills outcomes impact this activity singly or in conjunction 
with one or both of the other outcomes measures. 
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Academic progress and employment outcomes alone are most frequently 
named as impacting curriculum development (32%), course offerings (31%), 
services to "at risk" students (29%), and collaborations with business, industry 
and government (43%), and the miscellaneous set of all other acrivities and 
functions (32%). These figures mean that of all 591 mentions of curriculum 
development, for example, 32% of those responses were accompanied by a 
response indicating that it was impacted by only academic progress and 
employment outcomes. ^ p b 

Respondents naming accountability requirements, developing appropriate 
teaching and learning strategies, accreditation studies, and institutional planning 
in general most often mention that all three outcomes impact these activities 
and functions: 36%, 25%, 37%, and 36% respectively. For course scheduling, 
student satisfaction outcomes alone tie with the use of all three measures for 
impact (26%). 

Part C; Activities That Will Be Most Affected By Measnrps nf Student 
Outcomes * — 

Respondents were asked to do the same thing with respect to the next 3 -5 
years. Responses are rather similar to those given for the current situation. 
The number of responses is lower for the future expectation question, but the 
top two activities expected to be most impacted by these outcomes in the next 
3-5 years are the same named for the current situation, i.e., curriculum 
development and institutional planning. Development of appropriate teaching 
and learning strategies and accountability requirements replace course offerings 
and accreditation studies among the most frequemiy named activities that are 
expected to be influenced by outcomes measures in the future. 

Responses about outcomes that are expected to affect these outcomes in 
the future show that respondents generally do not think that any single set of 
outcomes measures will be a significant influence on these typical activities or 
functions but rather that all three sets of outcomes will influence them The 
exceptions to this general pattern are course placement, scheduling courses, and 
collaborations with business, industry, and government. Academic progress 
and employment outcomes are expected to replace student learning skills as the 
major type of outcomes measures to affect course placements, and this type of 
outcomes measure is expected to continue to be the primary influence on 
collaborations with business, industry, and government. Student satisfaction 
outcomes measures alone are expected by slightly more respondents than all 
three outcomes together to impact course scheduling activities. 
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Of particular interest is how respondents change the outcomes they expect 
to impact these activities ard functions relative to the current use of the 
outcomes. The change in course placement has already been mentioned. 
Improvements in academic advising and counseling; for another example, are 
less often expected to be impacted by student outcomes measures alone and 
more responses point to the influence of academic progress and employment 
outcomes measures alone or as one of these three sets of outcomes that will 
impact these efforts in the future. 

Summary 

In general, responses about which of these outcomes measures currently 
impact activities and functioits typical for the institutions in the survey 
compared to future expectations reflect findings from previous sections. 
Institutions today are more likelv to be using academic progress and employ- 
ment outcomes measures than either of the other types of outcomes measures 
examined in this study, but they expect to change that within the next 3-5 
years. Responses in this section indicate that institutions will be less likely to 
channel one set of outcomes measures to a specific office performing one or 
more of these activities or functions and in:reasingly more likely to gather 
measures on all types of outcomes to be shared across many offices oi the 
institutions. 
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Section V 



Oigaitization for Measurement of Student Outcomes 

V^^' ^i,"*^ section of the questionnaire was used to collect information about 
the admmistratiye organization of student outcomes measures. The goal is to 
detennme whether organizational attributes seem to be related to patterns of 
use for the student outcomes measures. This will be explored in more detail 
m Part n. The responses reported in this part will provide a general overview 
of whether colleges that participated in the survey have designated a position 
to be responsible for assessment, to tie student outcomes to institutional 
effertiyeness, how the position came about, problems associated with assessing 
mstituttonal effectiveness, and so forth. 

Respondents were first asked whether a person or position has been 
designated to direct or coordinate assessment for the purpose of assessing 
institutional effectiveness (see Item A). Over half of the insHmtions (55%) have 
identtfied a person for this function. Over a quarter of those (26%) call the 
position director or coordinator of institutional research, another 15% have the 
vice president or director of planning and development take this responsibility, 
ana about 8% have given the task to their dean of instruction. These are just 
the most frequently mentioned of many different position titles associated with 
this runction. 

Next, respondents were asked whether a person or position at their college 
has responsibility for directing or coordinaHng assessments of the kinds of 
student outcomes covered in previous sections of the questionnaire (see Item 
a7cr u u ^Institutions in the survey have assigned this responsibility: 
37% have the same person responsible for both student outcomes assessment 
and assessment of institutional effectiveness, and 14% have designated another 
person or posihon for this role. About 10% of these institutions have student 
outcomes assessment carried out by the chief student services officer, and 5% 
named the institutional research officer to this role. 

Tl\e person responsible for outcomes assessment most typically reports to 
he president (see Item C). Over half (53%) of the institutions having assigned 

^'^'i^' i'^''^'^^ P^'^" '^P°'' president, end about 16% 

report to the chief academic affairs officer. Further, three-quarters of the 
respondents mdicate that the president created the position (see Item D) Not 
3cT^7' .["^'^^H^^'^? have rather recently assigned this outcomes 

f.1f S?I^V^'PTi^?^'fy ^'^"^ ^"^^'^g 'hose having this position, one- 
half (53%) have had it less than three years. Twenty-six percent report having 
CTeated the position or designated the responsibility to a person within the last 
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year. On the other end, however, 59 institutions (17%) have had this role filled 
for at least ten years. 

Respondents report that several different factors initiated the process of 
assessing effectiveness at their college (see Item F). Seven factors were listed 
and respondents were given the opportunity to select any or all of those or to 
add any other factor to the list. Only 63 respondents gave no response to this 
question and 9% say they do no assessment of institutional effectiveness. The 
most frequently mentioned choice was "standard management practice." This 
was mentioned by 44% of the survey participants which is 49% of those who 
responded to this question. Activities related to curricular reform led 26% of 
the institutions in the survey to start assessing their effectiveness. Nearly as 
many (25%) opted for the "other" response and 18% said that the process was 
initiated in response to actions or policies of their accreditation board. The 
institutional governance structure initiated the process at 29% of the institutions. 
Academic reorganization and financial exigency were mentioned by about the 
same number of institutions-16% and 17%, respectively. In all, over 1200 
responses were given to this question for an average of about two initiating 
factors checked by each institution. 

Turning from what was responsible for initiating assessment of institutional 
effectiveness at the college to who, reveals considerable diversity (see Item G). 
Nearly a quarter (24%) chose the "not applicable" response indicating that the 
institution has no outcomes assessment systeia, per se. Almost as many 
institutions (21%) identify some office or department on campus as primarily 
responsible for the development of an outcomes assessment system. Examples 
given with this option include offices of institutional planning/ research, 
counseling, and admissions. Administrative committees/task forces and 
campus committees composed of administrators, faculty, and students were 
primarily responsible for this development at 17% of the institutions each. A 
faculty committee or task force was the primary catalyst at 11% of the colleges, 
and only 4% had the services of a consultant who played this role. 

The obstacles and impediments to assessing institutional effectiveness 
through measures of student outcomes such as those in the survey are of great 
interest. Faculty and staff in attendance at various presentations of preliminary 
results of the survey have been especially interested in this item, and they have 
strongly identified with the leading obstacles and impediments named by 
survey respondents. 

Thirteen options were available, and respondents could given as many as 
deemed appropriate (see Item H). Only 6% say they have encountered no 
obstacles, and 9% say that student outcomes are not used to assess institutional 
effectiveness at their college. 
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Among the specific obstacles or impediments, the leading one is lack of 
personnel resources. Over half of the institutions (52%) identify this factor, and 
lack of financial resources follows closely with 48% giving this as an obstacle. 
In addition, 31% find a lack of expertise to be an obstacle to accomplishing thio 
type of assessment. ^ 

Obstacles of a more technical character place in third and fourth position 
with resp^t to frequency of mention: lack of an adequate data base is a factor 
at 45% of the colleges and lack of adequate measurement instruments is 
mentioned by 39%. 

One-third of the respondents believe there is a general resistance to change 
on their campus. Similarly, 28% identify faculty resistance to outcomes 
assessment in pa-Hcular and 11% idenrify administrative resistance to outcomes 
assessment a? \c\es or impediments. Interestingly, student resistance or 
unwUlmgnc cooperate is not thought to be an obstacle by many institu- 
tions; only •/ , gave this response despite the very serious problem this has 
presented to institutions actually involved in outcomes assessment. 

The final section on the survey provides two scales similar to those used 
in each of the sections on outcomes measures (see Item I). Respondents were 
asked first to evaluate the effort of their institution in assessing effectiveness 
and then the effectiveness of that effort. It is quite possible for an institution 
to put fortii an outstanding effort but have low effectiveness associated with 
Uiat effort. On the other hand, an institution might have enjoyed a high level 
or^^^r^ctiveness in its assessment process despite a rather poor effort in that 

The mean effort score (1 = very poor; 5 = excellent effort) is 3.31 so effort 
tends to be rated higher than "neutral" or "neither poor nor good" and toward 
the good end of the scale. Indeed "good" was the modal response with 41% 
givmg this evaluation of effort. 

Effectiveness ratings were not as high. In fact, the difference in means 
IS statistically significant at the .000 level. The mean rating is 3.07--only slightly 
mgher than average. About equal numbers of respondents rated the 
effectiveness of their institution's assessment effort as very ineffpctive (29) as 
rated it effective (27). The highest number (254) of respondents gave' an 
average" effectiveness evaluation. 
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Summary 



Institutions having a person or position responsible for coordinat'on of 
assessment of institutional effectiveness and/or assessment of student outcomes 
are more common than expected. Over half of these institutions have one or 
both of these roleS/ and it is most typical to have the person or position reporl 
to the president. 

Few institutions initiated their process of assessing institutional effective- 
ness in response to accreditation board actions and policies, while most seem 
to view this as a step in keeping with standard management practices. 

The major obstacles to using student outcomes to assess institutional 
effectiveness are lack of personnel and financial resources. 

Institutions are getting higher marks for their overall effort in assessment 
than for the effectiveness of that effort. 
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PART II 



ANALYSES OF RELATIONSHIPS OF SPECIAL INTEREST 

Part II of this report covers a more in-depth analysis of the relationship 
of present and future priorities assigned to the use of these outcomes measures, 
how priority ratings and their expected changes are related to importance 
scores and use scores for the different sets of student outcomes, how certain 
institutional attributes are related to using outcomes measures for assessing 
institutional effectiveness, and finally, how the assignment of administrative 
responsibility for coordinating outcomes assessment and assessment of 
institutional effectiveness is related to attitudes and perceptions regarding the 
use of these assessments. 
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Section I 



Analysifi of Cunent and Future Priorities 
Introduction 

A number of topics covered in the survey raised interest and seemed to 
call for further analysis. Of particular interest is the question of what 
respondents perceive for the future use and role of these kinds of student 
outcomes. In this part of the report, current and future priorities will be 
examined in more detail to determine how they are related for each type of 
outcomes and how they relate across outcomes. Then priorities will be 
examined in relation to the importance respondents attach to the outcomes and 
according to the extent of use for each set of outcomes. All tables referred to 
in Part n are in Appendbc B. Tables 1-6 in Appendix B report the frequencies 
for current and future priorities. These were reported in Part I and are 
repeated here for easy reference. 

Changes in Priority Levels for Outcomes Measures Used to Assess Institutional 
Effectiveness During the Next 3-5 Years 

Academic Progress and Employment Outcomes 

o 38% expect priority assigned to academic progress and employment 
outcomes measures will increase within the next 3-5 years. Of that 
set of institutions, 26% are currently using these measures and expect 
the priority associated with their u?e to increase and 12% are not 
currently using these measures. Most of the institutions (59%) that 
are currently not using these measures expect them to have high 
priority within the next 3-5 years; 17% expect their use to be initiated 
in that time period and to receive the highest priority in assessing 
institutional effectiveness; 22% expect these measures to come into use 
but they do not expect them to receive high priority, and the 
remaining two institutions expect that these measures will be used 
but without any priority whatsoever. 

o 55% expect no change in priority from current use levels 

o 7% expect a decrease in the level of priority associated with this type 
of student outcome measure in the near future. 
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Student Learning Skills 



o 54% of the insHtutions indicate that they anticipate an increase in 
the level of priority associated with using measures of student 
learning skills for assessing institutional effectiveness during the next 
3-5 years. About half of these respondents expect an increase in 
prionty over current use levels, and the other half anticipate a shift 
from no use to usage with the following priority levels: 

29% the highest priority 

49% high priority 

21% not high 

1% no priority whatsoever 

o 42% expect no change in priority from current use levels. 

0 4% expect a decrease in priority associated with student learning 
skills outcomes measures. Sixteen of those 23 institutions are 
expected to decrease the outcomes from the highest to high priority 
and 6 expect to decrease priority from high to not high; only 1 
instihition expects a two-level decline from the highest to not high 
priority. ^ 

Student Satisfaction 

o 36% of the institutions expect an increase in priority associated with 
student satisfaction measures for assessing institutional effectiveness 
withm the next 3-5 years. Over one-half of those institutions (52%) 
do not currently use this type of outcome measure but expect to do 
so Within the next 3-5 years. The majority of those institutions (61%) 
will shift from no use to the high priority of use category, 14% expect 
o^S priority will be given to these outcomes measures, 

and 24% expect the priority to be associated with these measures will 
not be high. Only one institution is expected to start using these 
measures but accord no priority whatsoever. 

o 59% of the institutions anticipate no change from the current prioritv 
within tiie next 3-5 years. 

o 5% of the institutions that currently use measures of student 
satisfaction to assess institutional effectiveness expect a decrease in 
priority. 
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Relationships Between Current and Future Priority Levels 

Academic Progress and Employment Outcomes 

Institutions that currently place either high priority or the highest priority 
on the use of student academic progress and employment outcomes measures 
are most likely to anticipate that the same high level of priority will be given 
to the use of these measures during the near future (62% for highest current 
priority and 73% for high current priority). Of the institutions that currently 
do not use measures of academic progress and employment outcomes to assess 
institutional effectivene s, the majority (59%) anticipate future use with a high 
level of priority, 22% expect use of these outcomes to receive only low priority 
and 17% expect highest priority within the near future. Only 7% of respon- 
dents indicate that future priorities will be lower than the current level, and 
over three-quarter of those are at institutions that currently place highest 
priority on these outcomes measures. Further, of the institutions where current 
and future priority ratings are the same, over 90% are currently placing high 
or highest priority on these items (see Table 7, Appendix B). 

At institut* jns where the current situation is one of treating these outcomes 
as just another criteria in the assessment of institutional effectiveness without 
giving them nigh priority (level 2), the evidence for future behavior looks very 
promising: nearly three-quarters (72%) of those respondents assess future 
priorities as being either high or the highest, and none of the respondents 
indicate that the outcomes will drop further in status. 

Student Learning Skills 

Looking at the pattern of correspondence between the current priority level 
of student learning skills and the future level leads again to a sense of 
optimism (see Table 8). 

Of the 25 institutions in this portion of the analysis reporting that they 
currently place no priority whatsoever on using measures of student learning 
skills in the assessment of institutional effectiveniss, all expect the priority level 
to inaease within the next 3-5 years. 

Of the 146 institutions reporting that measures of these kinds of student 
learning outcomes are not currently collected, nearly half (49%) expect the 
future priority level to be characterized as "high" and over one-quarter (29%) 
anticipate these measures will be awarded the highest priority for accomplish- 
ing this purpose. ^The remainder of institutions currently not using these 
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outcx)mes anticipate that use will be made of these measures, but with no 
priority whatsoever being attributed these measures relative to otliers. 

Respondents at institutions where the priority levels associated with 
measures of student learning skills are already high or the highest anticipate 
maintaining high priority during the next 3-5 years. And, where change is 
j)ossible~given limits of the scale-the result will be a net gain in institutions 
that place highest priority on these items. No decrease in priority is expected 
for institutions that currently use student learning skills as a type of outcome 
measure to assess institutional effectiveness while placing priority lower than 
"high" on th6se measures relative to others. 

Forty-two institutions expect to change from non-users to highest priority 
users within the near future. That is, 29% of institutions that are currently 
non-users expect to be giving highest priority to the student learning outcomes. 
Four institiitions expect to shift from giving these student learning skills no 
priority whatsoever to placing the highest priority on them. Another 20 
institutions expect to move these outcomes measures into the highest priority 
category from the not high priority level of current use. Over one-third of the 
institutions that expect to give highest priority to these outcomes within the 
next 3-5 years will have rmed the priority by one level. 

Clearly, the trend is expected to be toward a higher use rate overall and 
a higher priority attached to the usage. 

Student Satisfaction 

Respondents who evaluate the priority of student satisfaction measures 
at th ir institution as \>eing high basically anticipate that level to be main'ained 
{66^oK but about one-third of those institutions that currently place the highest 
priority on these m.easures for assessing institutional effectiveness anticipate that 
the situation will change within the next 3-5 years so that the priority will 
decline by one level (see Table 9). Shifts into the highest priority level will, 
nonetheless, be sufficient to yield a net gain at the highest level. 

A majority (52%) of institutions currentiy at the second priority level, i.e., 
"not high," are expected to raise the priority of student satisfaction measures 
by one level to "high," and 9% of those anticipate a shift into the highest level 
of priority. 

Of those 10 institutions in this part of the analysis where no importance 
is currently placed on student satisfaction measures to assess institutional 
effectiveness, only one anticipates no change while half of them expect to be 
in the high priority category within the next 3-5 years. Finally, three-quarters 
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of the institutions that currently do not use student satisfaction measures for 
assessing institutional effectiveness expect to be giving high or highest priority 
to these measures over the next 3-5 years. 

Relationships Among Expecied Priority Changes for Each Type of Student 
Outcomes Measures 

A difference measure was computed the current priority levels from the 
future priority levels. The data reported in Table 10 shows the cross-tabulations 
for this difference in priority levels for academic progress and employment 
measures by measures of student learning skills. Of the institutions that expect 
no change in priority associated with academic progress and employment 
outcomes measures within tbo next 3-5 years, 53% also expect to maintain the 
status quo with respect to student learning skills measures, but institutions 
maintaining current priority levels on academic progress and employment 
outcomes represent 70% of those that expect no change in priority associated 
with stuc It learning skills outco»nes. This indicates that the future priority 
associated vvith student learning ; lis outcomes will be more active than that 
for academic progress and employment outcomes measures. 

It appears that measures of academic progress and employment outcomes 
will be slightly more dynamic in the near future relative to student satisfaction 
outcomes measures: 71% of the institutions that indicate the current priority 
level on academic progress and employment outcomes will be maintained also 
expect to maintain the current priority level on their use of student satisfaction 
measures for assessing institutional effectiveness. This is somewhat higher than 
the 66% of stable priority institutions with respect to student satisfaction 
measures that are expected to hold to current priority levels on academic 
progress and employment outcome^ (see Table 11). 

Relative to student satisfaction measures, measures of student learning 
outcomes are expected to experience far more change in the near future (see 
Table 12). Static institutions on the student satisfaction priority change measure 
account for 78% of those static on the student learning measures, but 
institutions static on student learning outcomes make up only 55% of those that 
expect to maintain current prior' ties on use of student satisfaction outcomes. 

Summary 

The respondents to this survey expect to see changes-and soon-in the use 
of these student outcomes measures for assessing institutional effectiveness. As 
previously reported, the trend during the next 3-5 years is expected to show 
increased use and increased priority associated with this use. What this section 
reveals more cleariy is that the priority associated with using student outcomes 
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measures of the types examined in this survey is not expected to decline. All 
will receive higher and higher priority in this use. Thero is the indication, 
however, that the use of student satisfaction outcomes in assessing institutional 
effectiveness may be leveling off during the next few years while other types 
of student outcomes wiU come more and more into use for this purpose. 

An interesting observation is that current priority levels are more diverse 
than priority levels expected during the next 3-5 years. This simply under- 
scores a very basic sense that there is considerable diversity in where we are 
now with respect to using student outcomes measures to assess institutional 
effectiveness, but there is little disagreement with where we want to be in the 
very near future. 
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Section II 

Relationships Between Priority of Use and Scores on Importance and Use 
Inde: es 

Introduction 

Having examined the relationships and inter-relationships of priority levels 
associated with student outcomes measures and how these are expected to 
change over the next 3-5 years, it will be interesting to determine whether 
priority levels are related to the importance scores attached to these items as 
indicators of institutional effectiveness and their actual use for that purpose. 
In this section findings are reported to compare importance and use index 
scores for each outcome type according to current priority ratings and then 
according to future priority expectations. Findings will also be reported that 
are based on analyses by differences among average importance index scores 
and among average use index scores computed for the different levels of 
current and future priorities. 

Importance Scores and Priorities 

The evidence suggests reason for frustration with the current situation, but 
there is also reason for optimism in the near future. Table 13 reports average 
importance scores for each set of student outcomes measures according to how 
respondents rate the priority currently associated with the set of measures. The 
statistics reported on this and subsequent tables are probably familiar. They 
are discussed in the methodology section (pages 15-16). 

The potential for frustration now is suggested by the fact that there is little 
relationship between the importance respondents place on student outcomes 
measures and their evaluation of the priority given these measures by their 
institution. To the extent that a relationship exists, it appears to be non-linear, 
particulariy for importance scores on academic progress and employment 
outcomes and stuclent learning outcomes. The current situation must be 
difficult for those respondents who place high importance on these measures 
as indicators of institutional effectiveness but who are part of an institution that 
either places no priority on that use or does not use the measures at all. That 
is evidenced by the fact that average importance scores do not systematically 
decrease with decreasing priority levels, although highest priority tends to 
correspond to highest importance scores. 

Turning to the future (Table 14), the results show that the higher 
importance ratings are associrted with higher priorities expected for the use of 
outcomes measures. Those few respondents that expect no priority to be 
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attached to student learning outcomes and academic progress and employment 
outcomes reverse this pattern, but the cases are too few to really alter the 
conclusions. 

Im portance Scores and Priority Changes 

The analysis of importance ratings according to changes in priority does 
not provide further insights into the relationship between the importance 
respondents assign to student outcomes measurss and their evaluation of the 
priority their institution gives those measures now and will give in the future 
(see Table 15). The results show that the differences in variances among means 
for two of the three types of outcomes measures is not significant. And, the 
findings for student satisfaction have no obvious explanation. The lowest 
importance scores, on average, are for institutions where priority levels are 
expected to increase by two or three levels in the next few years, and 
importance scores for institutions expecting no change equal those expecting 
maximum increases in priority, but those are exceeded by importance ratings 
of respondents expecting their institution to reduce priorities for the use of 
student satisfaction. 

Use Scores and Priorities 

The analyses of mean scores on use indices reveal general findings similar 
to those in the previous section on priorities. The relationship of use to current 
priorities is greater than that for use and future priorities. For current 
priorities (see Table 16), the findings suggest that the relationship between 
priorities and use is indeed fairly close. While the average use rate of 
academic progress and employment outcomes is greater than that for student 
learning skills, the relationship between priorities and use is stronger for 
sttident learning skills. About 20% of the variance in use of student learning 
skills can be accounted for by the priority levels attached to their use. For 
shident satisfaction measures, the outcomes set with the highest use rate, 
priority levels account for nearly one-third (31%) of the variance in use rates. 

Use scores and future priority levels have the strongest relationship, albeit 
only low, for student learning outcomes, and there is no significant relationship 
for academic progress and employment outcomes (Table 17). 

Use Scores and Priority Chang es 

Unlike the case for importance scores, changes in priority levels are related 
to use scores (see Table 18). As might be expected, the form of that 
relationship is for average use rates to decrease as increases in priority level 
changes increase. Those institutions that currently have highest use rates are 
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those where respondents are most likely to expect a decrease in priority. 
Conversely, institutions that cunently use relatively few measures in the sets 
of student outcomes dre expected to increase the priority-and, presumably 
usage~of these outcomes within the near future. The magnitude of the 
relationship between average use rates and changes in priorities is very nearly 
the same as that for current priority levels alone. 

Summary 

Little relationship is found between importance and current priority, but 
the relationship is a little higher for future priorities. Highest priority tends to 
be associated with highest importance ratings. There is not, however, a linear 
pattern of decreasing priority and lower mean importance ratings for either 
present or future priority ratings. In the case of future priorities, this seems to 
result from only very few respondents who expect to experience great 
dissonance with their institutions. 

Use scores and current priority ratings do show a fairly high level of 
relationship, the highest being with student satisfaction outcomes and the 
lowest with academic progress and employment outcomes. Future priority 
levels are only slightiy related to u:>e rates, and there is not a significant 
relationship for academic progress and employment outcomes. 

Examination of the relationship between change in current and future 
priorities and the importance and use scores for each set of outcomes shows 
that the importance scores are basically not related to the change in priority 
level while use scores are. 
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Section III 

Relationship of Institutional Attributes to Using Student Outcomes Measures 
to Assess Institutional Effectiveness 

Introduction 

Institutional attributes sucl\ as size of enrollment, regional accreditation 
association, and geographic setting-rural, urban, and so forth, are expected to 
be related to the importance perceived for these outcomes and their use in 
assessing institutional effectiveness. It seems likely that larger institutions differ 
from smaller ones in the kinds of outcomes they find important to measure 
and in the use made of these measures with respect to their assessment of 
institutional effectiveness. 

Similarly, the regional accreditation associations exhibit considerable 
variability with respect to the standards, guidelines, and requirements 
associated with using student outcomes measures for assessing institutional 
effectiveness. That being the case, it will be interesting to see whether 
institutions within the six regional accreditation associations tend to share 
similar use patterns and reflect similar attitudes and values regarding the use 
of student outcomes in assessing institutional effectiveness. 

A third piece of information was added to the data set so that responses 
could be analyzed in relation to this institutional attribute. An attribute that 
has proved useful at the 1989 Project Cooperation Sumn\er Conference for 
classifying institutions into fairly homogenous groupings is !;he "geographic 
setting." Geographic setting here refers to the site or locational ainbience--the 
milieu or environment in which the institution operates. This attribute seems 
to be h rough surrogate not only for size of institution and type and purpose 
of students, but of the role and purpose-mission-of the institution. Geograp- 
hic settings are coded as rural, city, suburban, and urban. The most 
ambiguous of these is "city." Institutions that are located in places with a 
population of around 25,000 but are dearly not a suburb are classified as "city." 
The addition of this coding group enhances the classification scheme over 
including these with suburban sites or simply using a random assignment. 

Enrollment Size 

Several groupings were experimented with for classifying institutions 
according to size of total enrollment. Total enrollment is a head count number 
of students in credit classes as of October 1987. (AACJC: 1988) The data 
presented on pages 122»1?.3 of Appendix B use four size groupings. 
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No matter which grouping is used, total enrollment size does not appear 
to be a significant, consistent indicator of attitudes and perceptions about and 
use of student outcomes measures for assessing institutional effectiveness. 



The results of analyzing the variance about importance score means for the 
four enrollment groupings show that there is a statistically significant difference 
for academic progress and employment outc(H:ies (Table 19). This is no doubt 
due to the smallest size category (1-1000 enrollments) since all other means are 
equal. Taken in isolation, this statistical result provides little support for a 
relationship between the importance associated with using student outcomes 
measures to assess institutional effectiveness and enrollment size of the 
institution. 

One might well argue that the importance scores are based on peiceptions 
of respondents rather than on a more concrete measure. While it is not clear 
that there is systematic bias in individual respondent's perception of importance 
by institutional size, there should be when actual usage is examined. Yet, the 
use scores are also unrelated to enrollment size (see Table 20). 

Since the results for the relationship of enrollment to responses throughout 
the survey generally show neither statistical significance nor any observable 
systematic pattern, those results are not reproduced here. The clear finding is 
that differences in perceptions about using and usage of student outcomes 
measures to assess institutional effectiveness are not related to the size of the 
institution. 

Regional Accreditation Association 

The second institutional attribute examined in relation to responses to this 
survey is the regional accreditation association with which the institution is 
affiliated. Institutions in the Southern Association of Colleges and Schools and 
the North Central Accreditation Association dominate responses to the survey: 

New England 4% 

Middle States 12% 

Southern 34% 

North Central 36% 

Northwest 1% 

Western 13% 

Again, responses to all questions in the survey were cross tabulated with 
the institution's regional accreditation association. Again, no statistically 
significant nor observable systematic pattern of relationship exists. 
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Data on the mean importance index scores and use index scores are 
reported in Tables 21 and 22. While the results for importance scores are not 
statistically significant, with one exception, they deserve further attention. 

Of these six regional accreditation associations, the Southern (SACS) stands 
out for its relatively strong stand on the use of outcomes measures for 
demonstrating institutional effectiveness. For none of the three sets of 
outcomes examined in this survey, however, is the mean importance score of 
the SACS institutions highest. Rather, the highest average importance ratings 
are given by institutions accredited by the Northwest Association. These high 
scores are perhaps true reflections of all institutions in the association, but the 
results are more likely due to the very small number of institutions from the 
Northwest Association that responded to the survey. These institutions that 
responded are probably the more intensely concerned institutions with respect 
to the general issue of assessing institutional effectiveness. 

There is a significant difference in use index scores across the six 
accreditation regions in both academic progress and employment outcomes and 
student satisfaction outcomes. Institutions in the Southern Association do have 
the highest average use score on student satisfaction outcomes and student 
learning outcomes, but the variation in use within accreditation regions is 
sufficiently high that this difference is not statistically significant. Further, the 
institutions from the Middle States Association report the highest average use 
of any of the associations. 

Geographical Setting 

The third institutional attribute thought to be associated with attitudes and 
perceptions about and uses of student outcomes measures for assessing 
institutional effectiveness is the geographical setting. 

Again, the findings reveal no observable systematic pattern of association 
between the geographical setting of institutions in the survey and responses to 
Items on the questionnaire. The results reported in Tables 23 and 24 in 
Appendix B show no statistically significant relationship or effect between 
geographical setting and the average ratings of either importance or use for any 
of the three types of student outcomes measures. 

Summary 

The expectation that these institutional attributes are related to attitudes 
and practices regarding the use of student outcomes measures to assess 
institiitional effectiveness is not substantiated. Indeed, the significant finding 
here is that nether enrollment size, regional accreditation associations nor 
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geographic setting is related to responses on this subject. Per'iaps it is too 
early tor these patterns to have formed with respect to use, but there is 
generally no pattern found in relation to importance ratings nor the question 
asking respondents to evaluate the priority that will be given these measures 
in the future. To date, none of these institutional attributes has significantly 
reduced diversity of thought and practice in the process of assessing institution- 
al effectiveness. 
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Section IV 



Relationship of Having an Assessment Coordinator to Using Student 
Outcomes Measures to Assess Institutional Effectiveness 

Introduction 

This is the final "special focus" section, and it examines the question of 
whether attitudes about, and perceptions, practices and patterns of using 
student outcomes measures to assess institutional effectiveness at institutions 
having a designated director or coordinator of assessment differ from those at 
institutions that have no such person or position. Institutions that have 
specified a person or position to be responsible for assessing institutional 
effectiveness and, perhaps, the same person or another to assess student 
outcomes such as those examined in the questionnaire are likely to use more 
of the outcomes, place higher priority on their use, rate the importance as 
greater, and to have integrated the measures into more of the standard 
activities and functions of the institution. 

In order to test this assumption, responses to two questions were 
combined into a single variable that is coded for institutions that have a coor- 
dinator/director of assessment who is responsible for assess/ng institutional 
effectiveness, but no one specifically responsible for measures of student 
outcomes; institutions that have designated a person or persons responsible 
both for the assessment of institutional effectiveness and/ or student outcomes; 
and institutions that have no coordinator or director for either purpose. 

As previously reported in Section I, over half (55%) of the institutions in 
the survey have a coordinator of assessment responsible for assessing 
institutional effectiveness, two-thirds of those (68%) are also responsible for 
assessing student outcomes. In fact, only slightly more than a quarter (27%) 
of the institutions that have a coordinator or director responsible for assessing 
student outcomes have given that responr ibility to a person other than the one 
in charge of assessing institutional effectiveness. Here is how that information 
translates into the new variable created for this analysis: 

Fifty-one percent of the institutions (344) have both assessment 
responsibilities (whether the same or different persons), 9 percent 
have designated an assessment coordinator for institutional effective- 
ness only, and 40% have neither (or did not respond). 
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Importance of Student Outcomes 

The now familiar index of importance scores is analyzed first. It appears 
that perceived importance (see Table 25) for the items included in each set of 
outcomes measures does not vary according to whether institutions have an 
assessment coordinator. Student satisfaction outcomes are an exception to this, 
however, and institutions that have designated someone to be responsible for 
both assessment of institutional effectiveness and student outcomes are those 
with the highest importance scores, on average, given to student satisfaction 
outcomes. 

Interestingly, the average importance score is exactly the same for student 
learning outcomes regardless of whether institutions have a coordinator of 
assessment for both institutional effectiveness and student outcomes, institution- 
al effectiveness only, or no coordinator at all. There is only slight difference 
in the average importance scores for academic progress and employment 
outcomes as well. 

Use of Student Outcomes 

Just as has been the case in previous analyses, importance scores show 
little or no relationship, but average use scores are significantly different, 
depending upon the presence and type of assessment coordinator the institution 
has appointed (see Table 26). For each type of outcomes measure the 
institutions that use the greatest number of items in the set, on average, have 
appointed a coordinator both for assessing institutional effectiveness and for 
assessing student outcomes. These institutions average using about twice as 
many student learning skills measures as those with no coordinator. For each 
outcomes type, the real difference in use rates is for institutions that have 
assigned both responsibilities; those with a coordinator of institutional 
effectiveness only have an average use rate about equal that of institutions with 
no assessment coordinator at all. 

Current and Future Priority of Outcomes 

For each type of student outcomes, institutions having designated someone 
to perform both assessment functions in questions are those from which 
institutions assigning highest priority and those assigning high priority are most 
like to come. And the rate of non-use is highest for institutions with no 
coordinator, no matter which outcome type is examined (see Tables 27-29). 

The statistics reported in Tables 27-29 show a significant, but rather low, 
relationship between the coordinator variable and current priorities associated 
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with each outcomcc set. This is not the case for the relationship with future 
priorities to be associated with the various outcomes measures studied here. 

As can be seen in Tables 30-32, the relationship between future priorities 
expected for the outcomes measures and assignments of assessment respon- 
sibilities is statisUcally significant only for student satisfaction outcomes. To the 
extent that any patten or systematic relationship can be observed, there is no 
immediately obvio'is explanation for that. For example, respondents from 
institutions that have no coordinator of assessments are more likely to anticipate 
that priorities in the future will be highest than respondents where there is an 
assessment coordinator for institutional effectiveness. For both academic 
progress and employment outcomes and student learning skills outcomes, 
mstitutions having assigned both assessment roles have the lowest rate of 
respondents expecting high priority to be associated with these outcomes of any 
of the three assessment coordinator configurations examined here. The 
percentage expecting high priority is the near future for student satisfaction 
outcomes is still higher for those institutions that have no assessment 
coordmator tha\i for those having both a coordinator of assessment for student 
outcomes and for assessing institutional effectiveness. 

gatis faction With Use of Outcomes 

Respondents were asked to rate the level of satisfaction at their college 
with how measures of the three types of student outcomes as used for 
assessing institutional effectiveness. This provides another summary measure 
of attitudes about the use of these outcomes, and it seems likely tha^ the 
satisfaction level will vary according to whether the institution hai> no 
CTOrdmator of assessment, a coordinator of assessments of institutional 
effectiveness only, or a coordinator of assessments of institutional effectiveness 
and student outcomes. 

The results are presented for the three outcomes types in Tables 3:^-35. 
The coordinator variable is significantly related to satisfaction with the use of 
each of the three kinds of student outcomes, and the relatio. .ship is especially 
clear with both academic progress and employment outcomes and student 
satisfaction outcomes. Just as one would expect, institutions with no 
coordinator are almost twice as likely to experience low and very low 
satisfaction with the use of student outcomes for assessing institutional 
effectiveness than those with a designated coordinator(s) for both types of 
assessment. And, inst'V Mons having a coordinator both of assessing institution- 
al effectiveness and stuuent outcomes are nearly twice as likely to experience 
very high satisfaction with the use of the assessments. This difference is miich 
less sharp for student learning skills outcomes. 
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Overall Rating of Effort and Effectiveness of Assessment 



The last items on the questionnaire ask the respondents to rate the overall 
effort of their institution with respect to using student outcomes measures for 
assessing institutional effectiveness and to also rate the effectiveness of that 
effort. These ratings are reported in Part I. In this final analysis section of the 
report, these ratings will be examined in relation to the assignment of 
assessment coordinator responsibilities. 

As with the satisfaction ratings just discussed, the expectation is that both 
effort and effectiveness will be more highly rated at institutions having 
designated a person(s) to direct assessment of both institutional effectiveness 
and student outcomes. The results reported in Tables 36 ana 37 verify this 
expectation, but with a slight twist. 

It would seem likely that institutional effort is less closely related to the 
assessnaent coordinator assignments or lack thereof than effectiveness. 
Institutions can exert great effort aimed at various activities and goals but the 
effectiveness of that effort may not be great, l^ui ticularly if there is no one to 
direct and coordinate the effort, one would expect the effectiveness to be low. 
But, the results point to a stronger relationship between the rating of 
institutional effort and the assessment coordinator assignment pattern than 
between coordinator and the effectiveness rating. 

Respondents from institutions having a coordinator(s) of both assessment 
of institutional effectiveness and of student outcomes are more than twice as 
likely to rate their institutional assessment effort to be good or excellent as 
those with no coordinator. And, respondents from institutions having assigned 
neither assessment responsibility examined here are almost three times more 
likely to give a poor or very poor effort rating than respondents where both 
assessment responsibilities are assigned. And, where assessment of both 
student outcomes and institutional effectiveness is assigned, effort ratings are 
more than twice as likely to be good or excellent. The same general pattern 
holds for the rating of effectiveness, but the magnitude of differences is not as 
great when effectiveness of the assessment effort is rated. 

Summary 

Responses were examined in this section according to whether the 
institution has appointed a coordinator responsible for assessment of both 
student outcomes and institutional effectiveness, a coordinator responsible for 
assessment of institutional effectiveness only, or no coordinator of assessment. 
Importance scores generally do not vary significantly with respect to the 
presence or type of assessment coordinator at an institution, but use scores do. 
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Of particular note is the fact that the difference for use scores is between 
institutions that have assigned both responsibilities versus those that have either 
no coordinator or a coordinator of institutional effectiveness only. 

The current priority assigned to using each set of student outcomes 
measures is rel=ited to the assignment of assessment coordinator responsibilities. 
These relationships are fairly low, especially for student learning skills 
outcomes. Further, for each set of outcomes, non-user institutions account for 
a higher percentage of the "no coordinator" category than the olher two 
assessment assignments examined. 

No systematic relationship is found for future priorities and the assignment 
arrangement of assessment responsibilities. 

Satisfaction with the use of these outcomes measures for assessing 
institutional effectiveness is related to whether an institution has assigned 
responsibility for assessing student outcomes and institutional effectiveness. 
The relationship is less when student learning skills outcomes are considered, 
however. 

Finally, institutions with no coordinator of assessment of student outcomes 
or institutional effectiveness are far more likely to get a low rating for their 
effort than institutions having assigned responsibility for both. Conversely, 
those with both assessment responsibilities assigned are far more likely to give 
high ratings to their effort. A similar, but somewhat less dramatic, pattern is 
found for the relationship between the coordinator variable and the respon- 
dent's rating of the effectiveness of the institution's effort in using student 
outcomes to assess institutional effectiveness. 
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A Survey of Student Outcome Measures 
Used to Assess Institutional Effectiveness 



Introduction 

The purpose of the survey is to determine whether members of AACJC are using 
student outcomes measures in assessing institutional effectiveness ^ which 
measures are used^ how they are being usedy and the effect of this use. 

While there are many different types of student outcomes to use in assessing 
institutional ef fectivenessy we have identified three for this survey: 
student academic progress and employment outcomes ^ student learning skill 
and student satisfaction* 

The survey has five sections, and we have found that it takes about 20-30 
minutes to complete. Please look over the survey and familiarize yourself 
with the different sections before you start to mark your responses. You are 
given the opportunity to describe measures not covered by the survey that are 
used on your campus. 

Your response to some items will vary according to the type of students you 
have in mind. We understand that no student body is totally homogeneous , but 
we ask that you think about your '^typical" student population and respond with 
that student type in mind. 



Now go to Section I 
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Section I: Studeut Academic Progress and Employment Outcomes Assessment 



1, 
2, 



This section explores measures of student academic progress and employment 
outcomes to determ- how these relate to assessing institutional 
effectiveness. 

At First, please evaluate the importance of each of these measures as an 
indicator of institutional effectiveness: NOT Important, MODERATELY 
Important, or VERY Important. You are asked to rate the importance even if 
your own institution does not use this measure. Then place a check in the 
space provided if your college uses this measure to assess its effectiveness. 

Level of Importance of Used to 

Academic Progress and Employment Assess 

Outcomes for Assessing Instit. 
Institutional Effectiveness Effectv. 

Not Moderately Very Check 
Important Important Important If Yes 



Number of students who graduate with an 
associate degree. 

Of those enrolled annually, the percentage of 
students who intend to complete the associate 
degree . 

Of those enrolled annually, the percentage of 
students who intend to transfer after earning 
the associate degree. 

Of those enrolled annually, the percentage of 
students who intend to transfer without 
earning the associate degree. 

The percentage of transfer students who receive 
a bachelors degree. 

The percentage of transfer students who do not 
receive a bachelors degree within a specified 
time period. 

The percentage of students who complete their 
intended program or degree at your institution. 

The percentage of students who do not complete 
their intended program or degree because they 
lefc for employment. 

9. The percentage of students in an associate degree 
program that earn the degree in three years. 

10. Grade point average of graduates. 

11. Grade point average of students who transfer 
before completing a degree program. 



4.0% 



16.9 



46.2% 



50.1 



49.0% 73.0% 



32.3 



31.6 



19.6 


46.8 


32.3 


34.1 


23.6 


49.8 


24.3 


22.8 


18.2 


40.9 


37.9 


23.6 


39.6 


41.0 


16.3 


11.6 


1.6 


20.9 


76.0 


58.8 


7.7 


42.1 


48.3 


37.3 


33.2 


46.7 


18. ;i 


19.1 


9.9 


54.7 


39.3 


23.0 


23.0 


51.4 


23.3 


19.0 
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Level of Importance of Used to 

Academic Progress and Employment Assess 

Outcomes for Assessing Instit^ 
Institutional Effectiveness Effectv# 

Mot Moderately Ve&y Check 
Important Important Important If Yes 



12. 


Grade point average of graduates who transfer 
relative to native students at four-year 
institution* 


lo.>}^ 






/. 0 CIV 


13. 


Grade point average of students who transfer 
without a degree relative to native students of 
the four-year institution. 


23.3 


40.4 


34.1 


27.9 


U. 


The percentage of graduates who find employment 
in their major/field of study. 




1 1 a 

1 1 .0 






15. 


The percentage of graduates who find employment 
not related to their major/field of study. 


7.9 


39.7 


51.0 


53.8 


16. 


Employer satisfaction with job training/skills 
enhancement courses. 


2.4 


9.5 


86.5 


59.7 


17. 


Employer satisfaction with graduates. 


1.5 


9.3 


88.3 


58.7 


18. 


First job earnings. 


16.0 


57.6 


24.6 


34.5 


19. 


Annual total income of graduates. 


28.4 


51.6 


18.5 


20.1 


20. 


Change and stability of career goals. 


"7.9 


55.6 


15.1 


9.2 


21. 


Career advancement of alumni. 


14.8 


56.0 


28.1 


17.0 



B. Please answer the following question(s) about the student academic 

progress and employment outcomes in the previous list. (Check all that 
apply.) 

11,9% Student academic progress and employment outcomes are not assessed 
by this institution. (Go to question E on page 4.) 

24.9 Student academic progress and employment outcomes are assessed, 

but they are not used to assess institutional effectiveness. (Go 
to Question 

61 .0 Student academic progress and employment outcomes are measured and 
used to assess institutional effectiveness. 



C. Next, please look back over this list of student academic progress and 
employment outcome data and select the item that is most important to 
assessing institutional effectiveness at your college today and the item 
that you believe will become the most important within the next 3'*-5 
years. Just enter the number (1-21) of each item in the spaces provided. 
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For example: If employer satisfaction with graduates is the most important 
Item in that list currently used for assessing institutional effectiveness 
at the college, enter 17 in the space. 

The most important of these measures currently used to assess 
institutional effectiveness at this college: 

1. Number of students who graduate with an 

associate degree. 9,3% 

2. Of those enrolled annually, the percentage of 
students who intend to complete the associate 
degree. 

3. Of those enrolled annually, the percentage of 
students who intend to transfer after earning 

the associate degree. ^ 2 

4. Of those enrolled annually, the percentage of 
students who intend to transfer without 

earning the associate degree. q^3 

5. The percentage of transfer students who receive 

a bachelors degree. ^ 5 

7. The percentage of students who complete their 

intended program or degree at your institution. 14.5 

8. The percentage of students who do not complete 
their intended program or degree because they 
left for employment. 

9. The percentage of students in an associate degree 

program that earn the degree in three years. 0.3 

10. Grade point average of graduates. 1^2 

11. Grade point average of students who transfer 

before completing a degree program. 0.3 

12. Grade point average of graduates who transfer 
relative to native students at four-year 

institution. ^ 2 

13. Grade point average of students who transfer 
without a degree relative to native students of 

the four-year institution. 0.9 

14. The percentage of graduates who find employment 

in their major/field of study. 26.5 

15. The percentage of graduates who find employment 

not related to their major/field of study. 0.4 

16. Employer satisfaction with job training/skills 

enhancement courses. 2 4 
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!?• Employei" satisfaction with graduatest 9.2 

18. First job earnings* 0.1 

20. Change and stability of career goals* OA 

21 « Career advancement of alumni. 0.1 

Inappropriate 11.9 

No response 11.4 

The most important of these measures that will be used to assess institutional 
effectiveness at this college within the next 3--5 years 

1* Number of students who graduate with an 

associate degree. 2.8% 

2. Of those enrolled annually, the percentage of 
students who intend to complete the associate 

degree 4 0.7 

3. Of those enrolled annually, the percentage of 
students who intend to transfer after earning 

the associate degree. 0.7 

4* Of those enrolled annually, the percentage of 
students who intend to transfer without 

earning the associate degree. 0.6 

5. The percentage of transfer students who receive 

a bachelors degree. 1.5 

6. The percentage of transifer students who do not 
receive a bachelors; degree within a specified 

time period. 0.1 

7. The percentage of students who conaplete their 

Intended program or degree at your institution* 20«4 

8. The percentage of students who do not complete 
their intended program or degree because they 

left for employment. 1.0 

9. The percentage of students in an associate degree 

program that earn the degree in three years. 0.7 

IO4 Grade point average of graduates- 0.3 

11 • Grade point average of students who transfer 

before completing a degree program. 0.1 

12* Grade point average of graduates who transfer 
relative to native students at four-year 

institution. 2.5 
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0.1 



13. Grade point average of students who transfer 
without a degree relative to native students of 
the four-year institution. 0.9 

14. The percentage of graduates who find employment 
in their major/field of study. oq.I 

15. The percentage of graduates who find employment 
not related to their major/field of study. 

16. Employer satisfaction with job training/skills 
enhancement courses. 

17. Employer satisfaction with graduates. 14.2 

18. First job earnings. 

20. Change and stability of career goals. 0.3 

21. Career advancement of alumni. i^q 

Inappropriate 11.9 

No response 12.2 

D. Relative to other measures used by this coll :ge to assess institutional 
effectiveness, how important are student outcomes items such as those in 
this section? (Check one) 

12^7% 1. These items are given tar higher priority than any others. 
^^'^ 2. These items are given high priority. 

15j_9_ 3. These items are just a part of the criteria used for assessing 

institutional effectiveness, but they are not high priority items. 

_0j6_ 4. Of the items used to assess institutional effectiveness, 

student progress and employment outcomes are least important and 
receive no priority whatsoever. 

11 '9 Inappropriate 
4.6 No response 

E. What do you expect the status of measures of outcome items such as those 
m this section to be during the next 3-5 years? (Check one) 

25jO% 1. These items will be given far higher priority than any others 
in the next 3-5 years. 

^1*9 2. These items will be given high priority. 

-IlL 3. These items will just be a part of the criteria used for 

assessing institutional effectiveness, but they are not given high 
priority. 
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0»4 4« Of the items used to assess institutional effectiveness during 

the next 3-5 years, this type of student outcome measures will be 
least important and receive no priority whatsoever^ 



3>5 No response 

F. On the ^cale below, please rate the level of satisfaction at, your college 
with how student academic progress and employment outcome measures are 
used there for assessing institutional effectiveness* 

Satisfaction with the Use of Academic Progress and Employment 
Outcomes Measures for Assessing Institutional Effectiveness 

3Z 17 • 8Z 44 • 6Z 24 • 71 3 . 4Z 

Very Low Medium High Very 

Low Satisfaction High 

Satisfaction Satisfaction 

No response=6«5% 

G. Are there other measures of student academic progress and employment 
outcomes that are used by your institution for assessing its 
effectiveness? If so, please identify those here. (Please attach 
additional sheets as needed and identify those as part of Section I.) 



Continue with Section II 
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Section II: student Learning Skills Assessment 

This section focuses on measures of studenL academic skills as a type of 
student outcome that may be used to assess institutional effectiveness. 

^'-^^"'''.^Jf^'! "^'^ importance of each of these measures of student 
ZlnT"" '^'l^'J'''' ^"'ff«i"8 institutional effectiveness. Remember that the 
importance of these skills is as an outcome measure and not as an income 
assessment. Vou are asked to rate the importance even if your ownl^FEItuti on 
."'^ these measures. Then place a check in the space provided if y.ur 
college does use this measure to assess its effectiveness. 



Foundation Skills 



Level of Importance of Used to 

Student Learning Skills Assess 

Measures for Assessing Instit, 
Institutional Effectiveness Effectv. 

Not Moderately Very check 
Important Important Important If Yes 



1. 


Reading Skills 


1.2% 


11.9% 


86.1% 


39.1% 


2. 


Writing Skills 


0.7 


11.7 


86.7 


42.4 


3. 


Oral Communication (speaking/listening) Skills 


2.5 


25.6 


70.5 


21.5 


4. 


Study Skills 


6.2 


40.4 


50.8 


13.8 


5. 


Arithmetic (computational) Skills 


1.5 


16.4 


80.6 


36.9 


6. 


Mathematics Skills (algebra) 


3.6 


37.2 


56.9 


28.9 


7. 


Advanced Mathematics Skills 












(trigonometry, calculus) 


22.4 


54.4 


18.8 


11.0 


8. 


Computer Literacy 


5.2 


47.6 


44.1 


12.4 



Process Skills 



9. Critical/Analytical Thinking Skills 


2.1 


19.3 


77.0 


16.0 


10. Synthesis/Integration Skills 


3.6 


33.8 


60.6 


9.6 


11. Self-Understanding Skills 


5.6 


44.3 


48.3 


8.1 


12. Aesthetic Appreciation Skills 


2.1 


11.4 


58.1 


6.2 


13. Social Responsibility Skills 


7.4 


44,6 


45.6 


7.9 


14. Life-Long or Self-Directed Learning Skills 


5.6 


36.0 


55.3 


7.3 


General Education/Field of Study Competency 








15. General Education Competency 


1.6 


23.3 


72.0 


25.5 


16. Field of Study Competency 


2.4 


17.9 


76.0 


30.5 
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Please i\nswcr the following question(s) about the student learning skills 
in the list above. (Check all that apply.) 



22.5% Student learning skills are not assessed by this institution. (Go 
to question F. ) 

Student learning skills are assessed, but they are not used to 
assess institutional effectiveness. (Go to Question C.) 
Student learning skills are measured and used to assess 
institutional effectiveness. 



43.0 



34.5 



C. Next, we would like to know about the methods used at your institution to 
measure student learning skills and the type of instrument you use in 
collecting these measures. 

For each student learning skill, please mark your response for Type of 

Assessment according to these instructions: 

o If the skill is not measured, leave it blank. 

0 If the skill is measured and the measure is used to assess 

institutional effectiveness, write a U for used in that space. 
0 If the skill i^ measured, but not used in assessing institutional 

effectiveness, please place a check ( ) in the space. 

Please check the Type(s) of Instrument(s ) you use. If it is a national 
instrument, please identify it by name or title. 

Measurement of Student Learning Skills 





Type 


of Assessment 




Type 


of Instrument 










Compare 




Dis- 






Name/Title 








Entry to 


Nati- 


trict/ 




Class- 


if National 




Entry 


Exit 


Exit 


onal 


State Campus 


room 


Instrument 


Foundation Skills: 


















1. Reading Skills 


















Measure 


63.1% 


19.3% 


12.6% 


33.5% 


7.0% 


16.6% 


11.0% 


16.6% = ASSET 


















10.7% = Mlsn-! 


Use 


14.4 


10.7 


6.4 


2.4 


1.8 


2.4 


1.9 




n 


523 


202 


128 


242 


59 


128 


87 


351 


2. Writing Skills 


















Measure 


58.4% 


21.5 


11.6 


22.4 


7.3 


25.2 


14.8 


13.3% = ASSET 


















4.1% = ACT 


Use 


14.4 


11.9 


6.5 


1.2 


1.5 


2.5 


2. A 




n 


491 


225 


122 


159 


59 


187 


116 


228 
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Type of Assessment 



Type of Instrument 











Compare 




Dis- 






Name/Title 








Exit 


Entry to 


Nati- 


trict/ 




Class- 


if NAtion/i1 


1 


Ural Conun* Skills 


Entry 


Exit 


onal 


State Campus 


room 


Instrument 




















7 n 


Q Q 




2.4 


1.0 


5.2 


12.3 


1.3% = ASSET 




Use 


0 7 


J.O 


1.3 


0.3 


0.1 


1.0 


1.8 




n 


65 


91 


17 


18 


8 


42 


95 


22 


A 

H» 


Study Skills 






















1 1 '\ 


R 0 
J %L 


1 n 

1.9 


4.0 


0.9 


5.5 


1*1 


2.5% - ASSET 




IIq^i 


J • u 


1 'i 


0.9 


0.1 


0.3 


0.6 


1.0 




n 


96 


44 


19 


28 


8 


41 


59 


38 


<\ 


Arithmetic Skills 




















Mpa cii^P 


• Q 


1 A 0 


y .2 


21 . J 


6.7 


21.3 


11.7 


15.6% = ASSET 




use 


iJ • u 


A 0 
O • 7 


3 .3 


1.5 


1.2 


2.4 


1.6 




n 


470 


74 


98 


154 


53 


160 


90 


44 


0 « 


Natnematic Skills 




















Mp/iqiit*p 


^7 


1 Q ^ 
lo • J 


y .5 


20.4 


7.3 


22.2 


12.0 


14.2% = ASSET 




IIqp 


1 0 Q 




3 .2 


1.6 


1.5 


2.2 


1.6 




n 


474 


186 


99 


149 


59 


1 ^ c 

165 


92 


42 


7 


AQV* Nat a oKlllS 




















Mp eiiT*0 




in 1 

iU • 1 


/. 

J 


9.8 


2.8 


11.4 


10.1 


8.1% = ASSET 




IIqp 






1 o 
1 .0 


1 f\ 
1.2 


0.3 


1.0 


1.0 




n 


216 


94 


35 


74 


21 


84 


75 


110 


8. 


Computer Literacy 




















Measure 


4.3 


8.0 


0.9 


0.9 


0.4 


5.2 


10.4 


0.3% = ASSET 




Use 


1 




0.3 


3.6 


0.1 


0.4 


1.2 




n 


^^7 


70 


0 

o 


6 


4 


38 


78 


6 


Process Skills 


















9. 


Critical /Analytical 




















Thinking Skills 




















Measure 


5.6 


9.3 


1.5 


3.7 






10.8 


2.0% = COMP 




Use 


L % L 


Q 

J • y 


1 0 

1 . 1 


0.7 


3.6 


0.3 


0.9 




n 




ay 


1 o 

18 


30 


6 


25, 


79 


39 


10. 


Synthesis/Inte- 




















gration Skills 




















Measure 


3.6 


6.5 


o.y 


1.6 


0.6 


2.2 


9.3 


2.0% = COMP 




Use 


1.6 


3.1 


1.2 


0.6 


3.6 


0.3 


0. 




n 


35 


65 


14 


15 


4 


17 


68 


39 
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Type of Assessment 



•r 



ypG of Instrument 









Compare 




Dis- 




Name/Title 








Entry to 


Nati- 


trict/ 


Class- 


if National 




Entry 


Exit 


Exit 


onal 


State Campus 


rooin 


Instrument 


Self -Under- 
















standing Skills 
















Measure 


4.3 


5.3 


0.6 


1.9 


0.4 2.4 


7.1 


1.4% = COMP 


Use 


0.9 


1.5 


0.7 


0.3 


3.6 3.6 


0.7 




n 


35 


46 


9 


15 


3 16 


53 


18 



12. Aesthetic Appre- 
ciation Skills 



Measure 


2.1 


4.7 


0.6 


1.2 


0 


1.2 


6.5 


0.9% = COMP 


Use 


0.6 


1.9 


0.6 


0.3 




0.3 


0.4 




n 


18 


45 


8 


10 


0 


10 


47 


14 



13. Social Responsi- 
bility Skills 



Measure 


2.5 


4.9 


0,7 


1.5 


0 


1.9 


5.9 


1.0% = COMP 


Use 


0.7 


1.6 


n.6 


0.3 




0.1 


0.7 




n 


22 


44 


9 


12 


0 


14 


45 


17 



14. Life-Long/Self- 
Directed Learning 
Skills 



Measure 


2.1 


3.6 


0.3 


0.7 


0.1 


2.1 


6.1 


0.4% = COMP 


Use 


0.3 


1.3 


0.4 


0.1 


0.1 


3.6 


0.6 




n 


16 


33 


5 


6 


2 


14 


45 


7 



General Education/Field 
of Study Competency 

15. General Education 
Competency 



Measure 


8.9 


11.7 


2.4 


4.9 


2.1 


5.6 


10.8 


3.4% = COMP 


Use 


3.4 


6.5 


2.1 


0.9 


0.4 


1.3 


1.0 




n 


83 


123 


30 


39 


17 


47. 


80 


58 



16. Field of Study 
Competency 



Measure 


5.5 


16.1 


1.6 


6.4 


6.8 


7.9 


15.1 


1.6% = NLN 


Use 


1.9 


7.6 


1.6 


1.0 


1.5 


1.3 


1.0 




n 


50 


160 


22 


50 


56 


62 


109 


29 



ERIC 



81 



8i 



Type of Student Learning Skills Assessment 
Results from Multiple Response Analysis 

Total 

# of Total # of 



Foundation Skills; 


Entry 


Exit 


Comparison 


Inst 


Responses 








!• Reading Skills 


61.3%* 


23.7% 


15.0% 


532 


853 


1. Writing Skills 


58.6 


26.8 


14.6 


512 


838 


3# Oral Comm. Skills 


37.6 


52.6 


9.8 


130 


173 


4. Study Skills 


60.4 


27,7 


11.9 


116 


159 


5# Arithmetic Skills 


63.3 


23.5 


13.2 


484 


742 


6# Mathematic Skills 


62.5 


24.5 


13.0 


AQA 


7 CO 

759 


7« AHv \As\th Qlri 1 1 e 






10. 1 


249 


345 


o. Computer Literacy 


31.6 


61.5 


6.8 


94 


117 


Process Skills 












9. Critical/Analytical 












Thinking Skills 


33.1 


55.6 


11.3 


120 


160 


10. Synthesis/Inte- 












gration Skills 


30.7 


57.0 


12.3 


84 


114 


11. Self-Under- 












standing Skills 


38.9 


51.1 


10.0 


72 


90 


12. Aesthetic Appre- 












ciation Skills 


25.4 


63.4 


11.3 


54 


71 


13. Social Responsi- 












bility Skills 


29.3 


58.7 


12.0 


56 


75 


14. L'fe-Long/Self- 












D ected Learning 












Skills 


29.6 


61.1 


9.3 


46 


54 


General Education/Field 












of Study Competency 












15. General Education 












Competency 


35.2 


52.1 


12.7 


167 


236 


16. Field of Study 












Competency 


21.6 


69.0 


9.5 


189 


232 



*% is computed as number checking "entry", for example, divided by the total 
number of responses given for a particular skill area. 
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X of Institutions Using Instruments 
To Assess Student Learning Skills 



ASSET ACT GAAP OOMP 

Foimdation Skills: 



1. 


Reading Skills 


32% 


8% 


1% 


1% 


2. 


Writing Skills 


40 


12 


2 


1 


3. 


Oral Comm. Skills 


41 


18 


5 


9 


4. 


Study Skills 


45 


13 


0 


n 

u 


5. 


A.rithmetic Skills 


43 


12 


1 


1 


6. 


Mathematic Skills 


40 


14 


2 


1 


7. 


Adv. Math Skills 


50 


13 


1 


0 


8. 


Computer Literacy 


33 


0 


0 


0 


Process Skills 










9. 


Critical /Analytical 
Thinking Skills 


10 


15 


13 


18 


10. 


Synthesi s/ Inte** 
gration Skill 


17 


28 


0 


28 


11. 


Self-Under- 
standing Skills 


6 


22 


0 


17 


12. 


Aesthetic Appre- 
ciation Skills 


14 


36 


0 


43 


13. 


Social Responsi- 
bility Skills 


12 


29 


0 


41 


14. 


Life-Long/Self- 
Directed Learning 
Skills 


14 


43 


0 


14 



CEEB STATE AAPPCC MAPS OTHER INSTITUTIONS 



6% 


7% 


6% 


5% 


34%^ 


351 


8 


7 


8 


6 


16 


228 


5 


5 


5 


5 


7 


22 


3 


0 


11 


5 


23 


38 


7 


7 


7 


7 


15 


244 


10 


7 


7 


6 


13 




7 


3 


5 


6 


15 


110 


0 


33 


0 


16 


16 


6 


10 


5 


3 


3 


23 


39 


6 


6 


6 


6 


4 


18 


0 


0 


6 


0 


50^ 


18 


0 


0 


0 


0 


7 


14 


0 


0 


0 


0 


18 


17 


0 


0 


0 


0 


29 


7 
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% of Institutions Using Instruments 
To Assess Student Learning Skills 

ASSET ACT CMP COMP CEEB STATE AAPPCC MAPS OTHER INSTITUTION!. 

General Education/ Field 

01 Study Competency 

15. General Education 

Competency 7 29 3 24 10 2 3 2 20 58 

16. Field of Study 

Competency 0 10 0 0 0 0 3 0 86-^ 29 

^ 21% of these use Nelson-Denny 
2 

22% of these use Myer-Briggs 

38% of these use the NLN (Nurse Licensure) 
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Next, please look back over this list of student learning skills and 
select the measure that is most important to assessing institutional 
effectiveness at your college today and the measure that you believe will 
become the most important within the next 3-5 years. 

For example: If a measure of oral communication skills is the most 
important measure in this list currently used for assessing institutional 
effectiveness at your college, enter 3 in the space. 

Most important of these measures currently used to assess 

institutional effectiveness at this college. 



Foundation Skil ls; 

1. Reading Skills 17.9Z 

2* Writing Skills 10.8 

3. Oral Comm. Skills 0.1 

5. Arithmetic Skills 0.4 

6. Mathematic Skills 0.7 

8. Computer Literacy 0.1 
Process Skills 

9. Critical/Analytical Thinking Skills 0.4 

10. Synthesis/Integration Skills 0.1 

13. Social Responsibility Skills 0.1 

14. Life-Long/Self-Directed Learning Skills 0.4 
General Education/Field of Study Competency 

15 o General Education Competency 6«4 

16. Field of Study Competency 13.4 

Inappropriate 22.5 

No response 26.2 

Will be the most importart of these measures used to assess 

institutional effectiveness at this college within the next 3-^5 years. 

Foundation Skills ; 

1. Reading Skills 10. 2Z 

2. Writing Skills 6.7 

3. Oral Comm^ Skills 0,7 
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5. Arithmetic Skills 0.3 

6. Mathematic Skills 0.6 

7. Adv. Math Skills 0.1 

8. Computer Literacy 1,5 
Process Skills 

9. Critical/Analytical Thinking Skills 7.6 

10. Synthesis/Integration Skills 0.9 

11. Self-Understanding Skills 0.1 
13 Social Responsibility Skills 0.3 

14. Life-Long/Self-Directed Learning Skills 2.8 
General Education/Field of Study Competency 

15. General Education Competency 15.3 

16. Field of Study Competency 12.5 

Inappropriate 22.5 

No response 17.8 



Relative to other measures used by this college to assess institutional 
effectiveness, how important are student learning skills items such as 
those in this section? (Check one) 

1. These items are given far higher priority than any others. 

2. These items are given high priority. 

3. These items are just a part of the criteria used for assessi 
institutional effectiveness, but they arc not high priority 
items. 

4. Of the items used to assess institutional effectiveness, 
student learning skills outcomes are least important and 
leceive no priority whatsoever. 

Inappropriate 

No response 



10.2% 
37.8 

hLzl 

J.il 

22.5 
8.3 




F. What do you expect the status of these measures in this section to be 
during the next 3-'5 years? (Check one) 



29. QX 1. These items will be given far higher priority than any others 

in the next 5 years • 

54«1 2. These itema will be given high priority. 

^^•^ 3. These items will just be a part of the criteria used for 

assessing institutional effectiveness, but they r not given 
high priority. 

0«6 4. Of the items used to assess institutional effectiveness during 
the next 3-5 years, this type of student outcome measures will 
be least important and receive no priority whatsoever. 

5«2 No response 

G. On the scale below, please rate the level of satisfaction at your college 
with how measures of student learning skills are used there for assessing 
institutional effectiveness. 

Satisfaction with the Dae of Student Learning Skills Measures 
for Assessing Institutional Effectiveness 

9 . 6% 29 . OX 33 . 9% 15 . 3Z 0 . 9Z 

Very Low Medium High Very 

Low Satisfaction High 

Satisfaction Satisfaction 

(No Response = 11.3%) 

H. Are there other measures of student learning skills that are used by your 
institution for assessing its effectiveness? If so, please identify those 
here. (Please attach additional sheets as needed and identify those as 
part of Section II.) 



Now go to Section III 
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Section III: student Satisfaction 



In this section, we ask for information about how measures of student 
satisfaction are used in assessing institutional effectiveness. 

A# Please evaluate how important it is lj include information on student 
satisfaction wiuh certain attributes of the institution in an assessment 
of institutional effectiveness. Circle the number that most accurately 
reflects the level of importance associated with each item of information 
in general — even if your institution does not include this in its 
assessment of effectiveness. Then place a check in the space provided if 
your college does use this measure to assess its effectiveness* 

Level of Importance of Usei To 

Student Satisfaction Measure 

Measures for Assessing Insti.t. 
Institutional Effectiveness Effer: v> 

Not Moderately Very Check 
I mportant Important I mportant If Yes 

1. Student Satisfaction with curriculum offerings 
in general, e.g. variety, quality, content, 

availability, class size 0.7% 23*0% 75.0% 51.0% 

2. Student satisfaction with courses in major 

area of study, e.g. variety, quality, content. 1.2 16.9 80.4 49.6 

3. Student satisfaction with academic support 
services, e.g. tutoring/learning centers, 

remedial courses, special counseling. 0.9 25.9 72.0 52*1 

4. Student satisfaction with faculty, e.g. quality, 

attitude toward students, availability. 0.4 15.4 83*0 58#7 

5# Student satisfaction with academic advising, e.g. 

quality, content, availability of advisor. 1.3 27.1 70.4 47.7 

6» student satisfaction with facilities, e.g. 

dormitories, classrooms, campus grounds 8#1 57.9 31.0 33#8 

7. Student satisfaction with services and service 
facilities, e.g. recreational, book store, 
parking, cafeteria, student union/commons, 

campus housing, student health, library. 4.1 57.0 37.2 40.6 

8. Student satisfaction with official procedures, 
e.g., admissions process, class registration, 
bill/fee paymenc, declaring/changing major of 

study^ applying for/receiving financial aid. 2.7 45.6 50.4 42.1 

9. S .udent satisfaction with career planning/skills 
*=• jhancement , e.g. preparation for chosen occupa- 
tion/job training, job skills enhancement. 1.5 36.1 60.6 40.0 
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Level of Importance of Used T'j 

Student Satisfaction Measure 

Measures for Assessing Instit# 
Institutional Effectiveness Effec tv> 

Not Moderately Very Check 
Importan t Important Important If Yes 

10. Student satisfaction with campus conditions in 
general, e.g# security and safety, racial 

harmony, 3,3% 52.4% 41.9% 26.5% 

11 • Student satisfaction with student development, 
efg# student role in governance, availability 

and variety of extracurricular activities, 5.8 59*4 32,6 32,3 

Please answer the following questiop(s) about the student satisfaction 
items in the previous list. (Check all that apply,) 

18*4% Student satisfaction is not assessed by this institution, (Go to 
question C, ) 

25.5 Student sat 'sf action is assessed, but the information is not used 
to assess institutional effectiveness, (Go to Question C) 

55,5 Student satisfaction is measured and used to assess institutional 
effectiveness, 

C, Next, please look back over this list of student satisfaction information 
and select the item that is most important to assessing institutional 
effectiveness at your college today and the item that you believe will 
become the most important within the next 3-5 years. Just enter the 
number (1-11) of each item in the spaces provided. 

For example: If student satisfaction with faculty is the most important 
item in that list currently used for assessing institutional effectiveness 
at the college, enter 4 in the space. 

Item // 

Most important of these measures currently used to assess 

insti ':ution£.l effectiveness at this college, 

1, Student Satisfaction with curriculum offerings 
in general, e,g# variety^ quality, content, 
availability, class size 27«6% 

Student satisfaction with courses in major 

i?f study, c.g variety, quality^ content, 12,9 

3, Studeiii: satisfaction with academic support 
services, e,g, tutoring/learning centers, 

remedial courses, special counseling, 1,0 
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4. Student satisfaction with faculty, e.g. quality, 

attitude toward students, availability. 21.6 

5. Student satisfaction with academic advising, e.g. 
quality, content, availability of advisor. 1.2 

6. Student satisfaction with facilities, e.g. 

dormitories, classrooms, campus grounds 0.1 

7. Student satisfaction with services and service 
facilities, e.g. recreational, bock store, 
parking, cafeteria, student union/commons, 

campus housing, student health, library. 0.3 

8. Student satisfaction with official procedures, 
e.g., admissions process, class registration, 
bill/fee payment, declaring/changing major of 

study, applying for/receiving financial aid. 1.5 

9. Student satisfaction with career planning/skills 
enhancement, e.g. preparation for chosen occupa- 
tion/job training, job skills enhancement. 2.8 

10. Student satisfaction with campus conditions in 
general, e.g. security and safety, racial 

harmony. 0.3 



Inappropriate 
No Response 



18. A 



12.3 



Item that will be the most important of these measures used 
to assess institutional effectiveness at this college 
within the next 3-5 years. 



1. 



Student Satisfaction with curriculum offerings 

in general, e.g. variety, quality, content, 

availability, class size 27. 7X 



2. 



Student satisfaction with courses in major 

area of study, e.g. variety, quality, content. 14.8 



3. 



Student satisfaction with academic support 
services, e.g. tutoring/learning centers, 

remedial courses, special counseling. 4,3 



4. 



Student satisfaction with faculty, e.g. quality, 

attitude toward students, availability. 13,0 



5. 



Student satisfaction with academic advising, e.g. 

quality, content, availability of advisor* 2.1 



6. 



Student satisfaction with facilities, e.g. 

dormitories, classrooms, campus grounds o.l 
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?• Student satisfaction with services and service 
facilities, e.g. recreational, book store, 
parking, cafeteria, student union/commons, 
campus housing, student health, library. 0.4 

8. Student satisfaction with official procedures, 
e.g., admissions process, class registration, 
bill/fee payment, declaring/changing major of 

study, applying f or/r*?.ceiving financial aid. 0.3 

9. Student satisfaction with career planning/skills 
enhance:iient , e.g. preparation for chosen occupa- 
tion/job training, job skills enhancement. 5.0 

10. Student satisfaction with campus conditions in 
general, e.g. security and safety, racial 

harmony • 0.1 

11. Student satisfaction with student deve:.opment , 
e.g. student role in governance, availability 

and variety of extracurricular activities. 0.4 

Inappropriate 18.4 
No response 13.1 

D. Relative to other measures used by this college to assess institutional 
effectiveness, how important are student satisfaction items such as those 
in this section? (Check one) 

10 > 5% 1. These items are given far higher priority than any others. 
48.3 2. These items are given high priority. 

16.3 3. These items are just a pirt of the criteria used for assessing 

institutional effecti" less, but they are not high priority items. 

1^3 4# Of the items used to assess institutional effectiveness, 

student satisfaction measures are least important and receive no 
priority whatsoever. 

18.4 Inappropriate 
5.0 No Response 

E. What do you expect the status of measures such as those in this section to 
be during the next 3-5 years? (Check one) 

16 > 3% 1. These items will be given far higher priority than any others* 
in the next 3-5 years. 

67 .0 2. These items will be given high priority. 
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12 t 6 3, These items will just be a part of the criteria used for 

assessing institutional effectiveness, but they are not given high 
priority, 

0>3 4. Of the items used to assess institutional effectiveness during 

the next 3-5 years, this type of student satisfaction measure will 
be least important and receive no priority whatsoever. 

3 t8 No Response 

F. On the scale below, please rate the level of satisfaction at your college 
with how these measures of student satisfaction are used there for 
assessing institutional ef fectiveness# 

Satisfaction with the Use of Student Satisfaction 
Measures for Assessing Institutional Effectiveness 

5.5Z 17. 2Z 37. 3X 28. 4Z 3. OX 

Very Low Med i um U i gh Ve r y 

Low Satisfaction High 

Satisfaction Satisfaction 

(8.6% No Response) 

G. Are there other student attitudinal data that are used by your institution 
for assessing its ef fectivenes? If so, please identify those here* (if 
you need more space, please attach additional sheets and identify those as 
part of Section III.) 



How go to Section IV 
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Section IV: Uses of Outcomes Assessment Information 

This section explores how your institution uses the various types of student 
outcomes information. We are assuming that the assessment of institutional 
effectiveness has a feedback effect so that various procedures and functions 
of the institution are impacted as a result cf the assessment process. 

A. The following list includes typical activities and functions of 

institutions such as yours, and we want to know the types of student 
outcomes information — if any-*-used in each of those activities or 
functions. 

Your response to Question B in the three previous sections is critical. 
Please make certain that you have marked your response to question B on 
pages 3f and 11. 

Place a check in the space if your institution uses the type of student 
outcomes measure in that column for the activity or function in this list. 





Academic Pro- 
gress and Employ- 
ment Outcomes 


Student 
Learning 
Skills 


Student 
Satisfaction 


1. Curriculum development 


71.1% 


44.1% 


41.0% 


2 , Create new positions 


37.8 


21.0 


13.8 


3. Course placement 


33.0 


52.6 


15.6 


4. Course offerings 


58.4 


41.0 


45.6 


5. Faculty development 


43.9 


31.1 


32.9 


6. Faculty incentives 


20.4 


9.5 


14.2 


7. Development of appropriate 
teaching/learning strategies 


44.4 


49.6 


36.4 


8* Funding purposes 


40.4 


22.1 


16.6 


9. Accountability requirements 


52.3 


29.3 


29.2 


10 Marketing 


53.9 


22.1 


54.7 


11. Improve academic advising/ 
counseling 


42.2 


37.5 


62.4 


12. Alternative instructional delivery 

modes (TV sites, Pl/CAl) 23.6 


21.8 


27.0 


13. Admissions procedures 


21.8 


21.5 


52.6 


14. Scheduling of courses 


33.5 


24.1 


58.2 


15. Resource allocation 


47.0 


30.7 


20.7 


16. Reporting (state and federal 
''unding) 


50.4 


23.6 


15.3 
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Academic Pro- 
gress and Employ- 
ment Outcomes 


Student 
Learning 
Skills 


Student 
Satisfaction 


17. 


Institutional planning, in general 72.0 


48.7 


55.9 


18. 


Accreditation studies/reports 


74.8 


52.4 


58.1 


X 7 • 


Student development 


29.5 


39.9 


51.0 


20. 


Services to "at risk" students 


39.0 


63.3 


37.6 


21. 


Promoting access to education 


36.3 


35.7 


37.6 


22. 


Economic and human resource 
development 


36.1 


19.4 


16.6 


23. 


Collaborations with secondary 
schools 


45.6 


38.7 


26.7 


24. 


Collaborations with other post- 
secondary institutions 


53.0 


29.2 


24.0 


25. 


Collaborations with business, 
industry, and government 


67.0 


29.9 


27.3 



Frequency Distribution for the Number of 

Activities and Functions Impacted 
Student Outcomes Measures and Functions 



X Institutions Using Outcomes: 





Academic Pro- 


Student 






gress and Employ- 


Learning 


Student 


or Functions 


ment Outcomes 


Skills 


Satisfaction 


0 


11.4% 


18.7% 


15.4 


1 


2.1 


4.7 


3.0 


2 


1.6 


3.4 


4.4 


3 


3.6 


4.4 


3.7 


4 


2.4 


4.9 


2.7 


5 


2.1 


4.9 


4.7 


6 


4.7 


4.1 


6.5 


7 


4.6 


4.3 


4.4 


8 


3.0 


5.9 


5.3 


9 


4.6 


3.4 


5.6 


10 


6.2 


4.3 


5.3 


11 


4.6 


4.1 


5.3 
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Z Institutions Using Outcomes: 




Academic Pro-- 


Student 




4 Act:ivit:ipfl or Piinrt'ionfl 


jjKtsGis auo Cinipi.ujr 


LieaFnxng 


btudent 


ment Outcomes 


Skills 


Satisfaction 


12 


5.3 


3.9 


5.2 


13 


3.0 


5.3 


3.9 


14 


4.1 


2.2 


3.7 


15 


5.3 


3.6 


4.6 


16 


6.1 


2.7 


3.7 


17 

X f 


4.' 


2.7 


2.7 


18 

X u 


3.0 


1.3 


1.9 


19 

X 7 


3.0 


2.2 


2.1 


21 


4.3 


1.8 


1.5 


21 


2.8 


2.7 


1.5 


22 


2.7 


2.1 


0.9 


23 


1.6 


0.9 


0.9 


24 


2.2 


0.6 


0.4 




1.6 


0.9 


0.6 


ki A A ^ 111 

Mean 11*3 


8.4 


8.7 




Median 11.0 


8.0 


8.0 




Standard Deviation 


7.1 


6.9 


6.4 



Frequency Distribution for the Number of 
Outcomes Measures Impacting 
Activities and Functions 



% Institutions Using t 
None of One Set Two Sets 



All Three 







Outcomes 


Outcomes 


Outcomes 


Outcomes 


1. 


Curriculum development 


17.6% 


32.6% 


25.6% 


24.1% 


2. 


Create new positions 


52.7 


28.1 


12.9 


6.2 


3. 


Course placement 


29.9 


45.8 


17.5 


6.8 


4. 


Course offerings 


23.3 


30.4 


24.4 


21.9 


5. 


Faculty development 


36.3 


33.3 


16.6 


13.8 
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X Inatitutions Usinj; ; 
None of One Set Two Sets All Three 
Outcomes Outcomes Outcomes Outcomes 



6. 


Faculty incentives 


72.0 


16.3 


7.3 


4.4 


7, 


Development of appropriate 
teaching/ learninf; strategies 


31.0 


28.1 


20.3 


20.6 


8. 


Funding purposes 


54.7 


22.2 


12.4 


10.7 


9. 


Accountability requirements 


42.2 


28.3 


16.0 


18.5 


10 


Marketing 


27.7 


30.7 


24.9 


16.7 


11. 


Improve academic advising/ 
counseling 


23.1 


32.9 


22.8 


21.2 


12. 


Alternative instructional delivery 
modes (TV sites, PI/CAI) 


56.9 


22.2 


12.6 


8.3 


13. 


Admissions procedures 


33.0 


46.2 


12.6 


8.0 


lA. 


licheduling of courses 


28.3 


41.0 


17.2 


13.5 


15. 


Resource allocation 


44.4 


26.8 


14.7 


14.1 


16. 


Reporting (state and federal 
fund i ni? ^ 


HO . H 


Zo.o 


1 /. c 

14 .5 


10.5 


17. 


T n Q ^ 1 ^ tl ^ 1 nni) 1 nlnnnincy r»anov*a1 
X iia u 1. u u i» i. uiia i. |^i.cl[l[li. , in gcnciTal. 


Z J . u 


ion 


16.6 


41,5 


18. 


Accreditation studies/reports 


21.3 


16.1 


18.4 


44.1 


19. 


Student development 


35.4 


28.1 


17.2 


19.3 


20. 


Services to "at risk." students 


23.3 


32.7 


24.9 


19.1 


21. 


Promoting access to education 


42.1 


24.9 


14.4 


18 / 


22. 


dllU iiUllloll LcaULfcCc 

development 


58.7 


21.5 


8.9 


11.0 


23. 


Collaborations with secondary 
schools 


37.2 


29.0 


19.4 


14.4 


24. 


Collaborations with other post- 
secondary institutions 


37.3 


33.0 


15.7 


13.9 


25. 


Collaborations with business, 
industry, and government 


28.1 


36.3 


18.8 


16.7 


B. 


From the previous list of 25 activities and 


functions , 


please select 


the 



uues. Lnat stana out ac your institution as having been most affected by 
the use of student outcomes measures. Write the item number(s) in the 
space provided. Also identify the type(s) of student outcomes measure 
(Academic Progress and Employment Outcomes, Student Learning Skills, and 
Student Satisfaction) associated with this activity or function. 
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1. Curriculum development 

2. Create new positions 

3. Course placement 

4. Course offerings 

5. Faculty development 

6. Faculty incentives 

7. Development of appropriate 
teaciiing/learning strategies 

8. Funding purposes 

9. Accountability requirement? 
10 Marketing 

11. Improve academic advising/ 
counseling 

12. Alternative instructional delivery 
modes (TV sites, Pl/CAI; 

13. Admissions procedures 

14. Scheduling of courses 

15. Resource allocation 

16. Reporting (state and federal 
funding) 

17. Institutional planning, in general 

18. Accreditation studies/reports 

19. Student development 

20. Services to "at risk" students 

21. Promoting access to education 

22. Economic and hum,in resource 
development 

23. Collaborations vith secondary 
schools 

24. Collaborations with other post- 
secondary institutions 

25. Collaborations with business, 
industry, and jjovernment 
Average 



Mentioning 
First 

27.9% 

0.7 

6.8 

6.2 

1.5 

0.3 

3.6 
1.8 
3.4 
3.4 

3.1 

0.1 
0.4 
1.0 
1.6 

1.0 
6.2 
3.1 
0.6 
1.5 
0.9 

0.6 

0.0 

0.7 

0.7 



X Mentioning 
Second 

5.8% 

1.8 

4.3 

8.9 

3.6 

0.1 

4.3 
1.5 
3.0 
4.0 

6.1 

0.7 
1.6 
2.7 
1.9 

2.5 
5.9 
5.0 
0.7 
2.7 
0.7 

0.4 

0.6 

i .0 

2.2 



Z Mentioning 
Third 

3.4% 

0.0 

0.7 

4.1 

1.5 

0.0 

3.3 

:.2 

1.3 
3.6 

4.3 

0.4 
1.5 
3.4 
2.7 

2.2 
9.2 
5.9 
2.4 
5.8 
1.6 

0.3 

1.9 

1.3 

4.6 
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Question B continued 



Order of Response 

1st item mentioned 
2nd item mentioned 
3rd item mentioned 



Academic Progress 
and Employment Outcomes 

44.9% 

42.5 

38.5 



Student 
Learning Skills 

29. 5X 

28.0 

27.0 



Student 
Satisfaction 

28.6% 

28.0 

29.3 



PLEASE TURN TO THE NEXT PAGE FOR THE TABLE REPORTING 
MULTIPLE RESPONSE FREQUENCIES 



C. 



What do you anticipate for the next 3-5 years? Which activities and 
functions, will be most affected by the use of student outcomes measures at 
your institution? 



Z Mentioning Z Mentioning 
First Second 



1. 
2. 
3. 
4. 
5. 
6. 
7. 

8. 
9. 
10. 
11. 

12. 

13. 
14. 
15. 
16. 



Curriculum development 

Create new positions 

Course placement 

Course offerings 

Faculty development 

Faculty incentives 

Development of appropriate 
teaching/learning strategies 

Funding purposes 

Accountability requirements 

Marketing 

Improve academic advising/ 



counseling 

Alternative instructional delivery 
modes (TV sites, Pl/CAI) 

Admissions procedures 

Scheduling of courses 

Resource allocation 

Reporting (state and federal 
funding) 



24.4% 
0.7 
1.9 
3.4 
2.7 
0.1 

5.9 
2.7 
4.7 
2.7 

1.8 

0.3 
0.3 
0.6 
2.5 

1.2 



17. Institutional planning, in general 10.1 



5.9% 

1.6 

2.4 

5.2 

5.2 

0.1 

5.2 
1.5 
4.4 
3.7 

4.3 

0.7 
0.7 
2.4 
3.6 

i.6 
6.8 



Z Mentioning 
Third 

5.0% 

0.1 

0.7 

2.2 

2.2 

0.4 

4.9 
1.2 
4.4 
3.9 

3.7 

1.2 
0.6 
1.5 
3.3 

1.6 
9.0 
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MULTIPLE RESPONSE FREQUENCIES FOR PATTERNS OF OUTCOMES: 
FUTURE EXPECTATIONS FOR MOST AFFECTED ACTIVITIES BY OUTCOMES MEASURES AFFECTING THEM 



Outcomes Measures Currclr Course Course Tching Accntblty Acad. Course Inst Accrdt "At Risk" Srvc to All 

Use Pattern D vlpmnt Plcmnt Offrngs Strtgs Rgrmnts Advsng Schdlng Plnng Stndrds Stndnts Bsnss Qthrs 

AP & EO Only 29.73: 30. 3Z 32. 6Z 28.1% 30.5% 20.4% 10.9% 22.2% 26.4% 26.7% 45.3% 34.1% 
(n = 902; 29.9%) 

SLS Only 9.2 25.8 9.4 8.5 5.6 10.9 3.1 5.1 7.1 17.8 9.4 9.2 

(n « 279} 9.3%) 

SS Only 9.8 7.6 13.0 4.5 5.6 17.5 34.4 7.6 2.9 10.4 7.9 12.1 

(n 303; 10.1%) 

Both AP & EO 

and SLS 7 .0 6.1 3.6 10.3 9.0 2.9 4. 7 5.4 6.4 7.4 7.2 5.7 

(n = 189; 6.3%) 

Both AP & EO 

and SS 4.9 9.1 3.6 4.5 6.2 6.6 7.8 4.9 7.9 6.7 6.5 5.4 

(n = 165; 5.5%) 

Both SLS and SS 2.1 3.0 4.3 5.8 1.7 8.8 7.8 4.3 1.4 2.2 1.4 2.0 

(n = 93; 3.1%) 

All three outcomes 

measures 37.3 18.2 33.3 37.9 41.2 32.8 31.3 50.4 47.9 28.9 22.3 31.5 

(n « 1083; 35.9%) 

Number of Responses 512 66 138 224 177 137 64 369 l40 135 139 841 

(n = 3014; 100%) 17.0% 2.2% 4.6% 7.4% 5.9% 4.5% 2.1% 12.2% 4.6% 4.5% 4.6% 27.9% 
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Z Mentioning S Mentioning X Mentioning 
Firsi. Becond Third 



18* Accreditation studies/reports 

19. Student development 

20. Services to "at risk" students 

21. Promoting access to education 

?2. Economic, and human resource 
development 

23. Collaborations with secondary 
schools 

24. Collaborations with other post- 
secondary institutions 

25. Collaborations with business, 
industry, and government 
Average 



2.4 
0.9 
1.6 
1.0 

1.0 

0.7 

0.4 

1.8 



3.6 
1.5 
3.0 
1.2 

1.5 

0.9 

0.9 

3.0 



Order of Response 

1st item mentioned 
2n»i item mentioned 
3rd item mentioned 



Question C continued 

Academic Progress Student 
and Employment Outcomes Learning Skills 



41.6% 
37.8 
36.3 



31.4% 
30.1 
27.6 



3.9 

1.9 
4.6 
1.3 

1.5 

2.2 

1.3 

5.2 



Student 
Satisfactiot? 

27.4% 
25.9 
27.9 



PLEASE TimN TO THE NEXT PAGE 
FOR THE TABLE REPORTING MULTIPLE RESPONSE FREQUENCIES 

D. If we have omitted some activities and functions at your institution that 
have been effected by the use of student outcomes measures, please 
identify those in the space below and provide a brief description. (If 
more space is needed, attach additional sheets, and identify them as 
Section IV.) 



Please Continue with the Final Section 
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MULTIPLE RESPONSE FREQUENCIES FOR PATTERNS OF OUTCOMES- 
FUTURE EXPECTATIONS FOR MOST AFFECTED ACTIVITIES BY OUTCOMES MEASURES AFFECTING THEM 

Outcomes Measures Currclr Course Course Tching Accntblty Acad. Course Inst Accrdt "At Risk" Srvc to All 

Use Pattern Dvl£tnnt Plcrnnt Offrngs Strtgs Rgrmnts Advsng Schdlng Plnng Stndrds Stndnts Bsnss Othrs 

(n = 902r29.9%) ^^'^^ ^^'^^ ^^'^^ ^^''^^ ^^'^^ ^^'^'^ ^^.3% 34. U 

(n'=T79; 9.3%) '''' ^7.8 9.4 9.2 

(n =1^3; 10.1%) ^^-^ 12.1 
Both AP & EO 

(",6.3%) '-^ 3.7 
Both AP & EO 

(nVL, 5.5%) 5.4 

(r=93-3a%r ^'^ ''^ '-^ ''•^ 2.2 1.4 2.0 

All three outcomes 

(n"3i 35.9%) "'^ '^'^ ""^ ^^-^ ^'-^ 28.9 22.3 31.5 

Number of Responses 512 66 138 224 177 137 64 369 14n it; \ia a/i 

(n = 3014, 100%) 17.0% 2.2% 4.6% 7.4% 5.9% 4.5% 2.1% 12.2% ^.6% 4.5% 4.6% 21.9% 
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Section V: Organ^zatioIIl for Measurement of Student Outcomes 



In this final section we ask how the outcomes measurement process is carried 
on in your institution. This information will be used to determine whether 
certain organizational patterns are associated with specific assessment 
activities and which organizational patterns seem to be associated with more 
effective institutions. 

A. Is there a person or position at your institution who is designated to 
direct or coordinate assessment for the purpose of assessing institutional 
effectiveness? 

Yes Please give the title (these are most frequently named;) 

26.4% Director/Coordinator Institutional Research 
14.6 VP/Director Planning and Development 
7.5 Dean of instruction 

1.6 No 

B. Is there also a person or position repsonsible for directing or 
coordinating the assessment of the types of student outcomes examined in 
this survey? 

37.2% Yes, it is the same person/position. 

^3.8 Yes, there is a different person/position responsible for that. 

^•'3 Mo, no one has specifically been given that duty. (Skip to item F.) 
Please give the title of this position. 9.5% Chief Officer of Student Services 

C. To whom does this person/position (refered to in item B) report? 

52.9% ^ 1. President 

1^5.7 2. Vice President Academic Affairs 

5.2 3. Vice President Student Affairs 

5«2 4. Assistant Vice President/Dean, Academic Affairs 

3«2 5. Assistant Vice President/Dean, Student Affairs 

^^'8 6. Other (please specify) (these are mo st frequently named) 

3.8 VP/Dean "general" or of both Student Affairs 
and Academic Affairs 

2.9% VP Administration 

49.0% Inappropriate or No response 
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D« Who was primarily responsible for initiating the creation of this position 
or assigning this responsibility? 



74.5% 


1. 


President 


9.6 


2. 


Vice President Academic Affairs 


2.0 


3. 


Vice President Student Affairs 


3.5 


4. 


Some other higher-level administrator 


U • 0 


c 

J • 


V acui cy 






Du UUcIlL S 


3.2 


7. 


Trustees of the College 


1.2 


8. 


A regional/district-level governing agent/agency 


2.0 


9. 


A state-level governing agent/agency such as Governor, State 
Superintendent, State Secretary of Education, or the like 


1.6 


10. 


Other (please specify) 1.8% Accrediting Agency 


49.0% 




Inappropriate or No response 



E# How long has this position or responsibility been defined? 
(check only the most appropriate) 



2.9% 


1. 


it has been approved/assigned but not 


25.9 


2. 


less than one year 


14.4 


3. 


less than two years 


10.3 


4. 


less than three years 


17.8 


5. 


three to five years 


11.8 


6. 


more than five years, less than ten 


17.0 


7. 


at least ten years 


49.0% 




Inappropriate or No response 
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F. Under what circumstances was the process of assessing institutional effectiveness 
initiated? (Check all that apply.) 



16. 7X 


1. 


financial exigency 


26.4 


2. 


curricular reform 


16.3 


3. 


academic reorganization 


31.0 


4. 


state officials 


20.0 


5. 


institutional governance structure 


44.3 


6. 


standard management practice 


9.3 


7. 


we do not assess institutional effectiveness 


24.7 


8. 


other (please specify) 17.5% Accreditation Board 



l%8% = Recruitment 

G. Who was primarily responsible for the development of your outcomes assessment 
system? 

lOjSt 1. faculty committee (task force, etc*) 
^7>4 2. administrative committee (task force, etc.) 
3*9 3. consultant 
16^ 4. a campus committee composed of faculty, adriinistration and students 

20^ 5. an office or department on campus, such as institutional 
planning/research, counseling, or admissions 

24j_l 6. not applicable; we do not have an outcomes assessment system 

^'Q 7. other (please specify) 0.1% State Governing Board 

0.3 No system in place, just plans 

H. Which of the following obstacles or impediments to assessing institutional 
effectiveness through the sorts of student outcomes measures covered in 
this survey were major ones at your institution? (Check all that apply) 

33. 4 X 1. general resistance to change 

2^*0 2. faculty resistence to outcomes assessment 

^^•2 3. administrative resistence to outcomes assessment 

7.0 4. lack of student willingness to cooperate 

48.4 5. lack of financial resources 

51*8 6. lack of personnel resources 

30.8 7. lack of expertise 

44.6 8. lack of adequate data base 

24»0 9. lack of adequate data processing capacity 
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10. lack of adequate measurement instruments 
11 • none, we have encountered no obstacles. 

12« not applicable; student outcome measures not used for that 
13. other (please specify) 0.9% Inertia; difficulty of change 

0.7 too early to tell 

I. Finally, we would like to get your overall rating of your institution's 
process of assessing effectiveness. We will ask you to evaluate the 
effort first and then the effectiveness of that effort. 

You may feel that the effort is outstanding, but the results, to date, 
would lead you to evaluate the effectiveness of the effort as being very 
low. You might feel just the opposite, that the effectiveness is quite 
high, despite a rather poor effort directed at this. 

Two different scales are provided here for you to mark (circle) your 
evaluation of etfort and effectiveness. 



38.8 
5.8 
8.8 
5.1 



Effort to Assess Institutional Effectiveness at This College 

3 .6% 17 . 9% 24.6% 40 . 7% 6 . 7% 

Very Poor Neutral/ Good Excellent 

Poor Neither 



Effectiveness of Assessment Efforts 

4.3%— 18.5% 37.6% 25.5% 4.0% 

Very Moderately Average Moderately Very 

Inef feet ivelnef feet ive Effective Effective 



The Final Page is Optional 



ERIC 



105 



109 



Optional 



Is there any '«iy in which AACJC, one of these councils (NCIA or NCSD), or ACT 
can be of helj. uO you in the area of assessing institutional effectiveness? 
If so, please give a brief description of this here. 



Please give your name, or the name of someone to contact regarding this 
matter. 



(Name) 

(Title) 

(Institution Name) 

(Street Address) 

(City, State, Zip) 
(Telephone) ( ) 



Thank you for your assistance! 
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APPENDIX B 




u 



Table 1 

Priority Currently Associated with Academic 
Progress and Employment Outcomes Measures 



Highest priority 12.7% 

High priority 54.1 

Not high priority 15.6 
No priority 0.1 

Not used 11.6 
No response 5.5 



Table 2 

Priority Currently Associated with Student 
Learning Skills Outcomes Measures 



Highest priority 10.2% 

High priority 37.6 

Not high priority 17.5 

No priority 3.7 

Not used 21.6 

No response 9.3 



Table 3 

Priority Currently Associated with 
Student Satisfaction Outcomes Measures 



Highest priority 10.4% 

High priority 48.2 

Not high priority 16.2 
No priority 1.5 

Not used 17.8 

No response 6.1 
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Table 4 



Priority Expect to be Associated with 
Academic Progress and Employment Outcomes Measures 
Within the next 3-5 Years 



Highest priority 24.4% 

High priority 60.9 
Not high priority 8.7 
No priority 0.4 
No response 5.5 



Table 5 

Priority Expect to be Associated with 
Student Learning Skills Outcomes Measures 
Within the Next 3-5 Years 



Highest priority 28.2% 

High priority 51.3 

Not high priority H.O 
No priority 0.3 
No response 93 



Table 6 

Priority Expect to be Associated with 
Student Satisfaction Outcomes Measures 
Within the Next 3-5 Years 



Highest priority 16.o% 

High priority 65.8 

Not high priority II.9 
No priority 0.3 
No response 6!l 
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Table 7 



Future Priority 
Level 



Relationships Between Current and Future 
Priority Levels for Academic Progress 
and Employment Outcomes Measures 

Current Priority Level 
Hi ghest High Not High None 



Not Used n 



Highest 
High 
Not high 
None 



6U% 


24.1% 


10.5% 


0.0% 


16.7% 


165 


36.0 


73.4 


61.0 


50.0 


59.0 


411 


2.3 


2.5 


28.6 


25.0 


21.8 


59 


0.0 


0.0 


0.0 


25.0 


2.6 


3 


n = 86 


365 


105 


4 


78 


638 



r = .34 

eta, future priority dependent = .42 
eta, current priority dependent = .35 

Cramer's v = ,33 



Table 8 

Relationships Between Current and Future 
Priority Levels for Student Learning Outcomes Measures 



Future Priority 
Level 



Current Priority Level 
Highest High Not High None 



Not Used n 



Highest 
High 
Not high 
None 



75.4% 


28.3% 


16.9% 


16.0% 


28.8% 


190 


23.2 


69.3 


59.3 


52.0 


48.6 


346 


1.4 


2.4 


23.7 


32.0 


21.2 


74 


0.0 


0.0 


0.0 


0.0 


1.4 


2 


n = 69 


254 


118 


25 


146 


612 



r = .27 

eta, future priority dependent = .38 
eta, current priority dependent = .30 

Cramer's v = .28 
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Table 9 



Relationships Between Current and Future 
Priority Levels for Student Satisfaction Outcomes Measures 

Future Priority Current Priority Level 

Lgyel . Highest High Not High None Not Used 



Highest 
High 
Not high 
None 



r = .32 

eta, future priority dependent = .47 
eta, current priority dependent = .34 

Cramer's v = .39 



65.7% 


10.5% 


9.2% 


10.0% 


14.2% 


32.9 


88.0 


52.3 


50.0 


60.8 


1.4 


1.5 


38.5 


30.0 


24.2 


0.0 


0.0 


0.0 


10.0 


0.8 


n = 70 


325 


109 


10 


120 



112 



1V6 



Academic Progress 



and Employment Declines 

Outcomes 2 Levels 

Declines 2 Levels 0.0% 

Declines 1 Level 0.0 

No Change 0.0 

Inaeases 1 Level 0.0 
Increases 2 Levels 0.0 

Increases 3 Levels 2.4 

100.0 

Increases 4 Levels 0.0 



n = 1 

r = .24 
eta = .27 
Cramer's v = .17 
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Table 10 

Relationship Between Expected Priority Change 
for Academic Progress and Employment Outcomes 
and Student Leamiri^: Skills 



Priority Change in Student Learning Skills 



Declines 


No 


Increases 


Increases 


Increases 


Increases 








1 Level 


z Levels 


o Levels 


4 Levels 


n 


0.0% 


50.0% 


0.0% 


50.0% 


0.0% 


0.0% 


2 


5.3 


34.2 


44.7 


5.3 


5.3 


5.3 


38 


4. n 


DO.'i 


1Q 1 


Q ^ 

y.o 


o o 


4.3 


324 


61.9 


69.5 


43.1 


46.9 


45.7 


35.0 




2.8 


29.6 


35.9 


12.0 


10.6 


9.2 


142 


0.0 


27.6 


13.8 


24.1 


24.1 


10.3 


29 




3.2 


6.3 


10.9 


14.3 


10.0 




4.9 


19.5 


22.0 


17.1 


24.4 


9.8 


41 


9.5 


3.2 


6.3 


10.9 


14.3 


10.0 




0.0 


30.8 


7.7 


0.0 


30.8 


30.8 


13 




1.6 


0.7 




5.7 


10.0 




21 


249 


144 


64 


70 


40 


589 



Table 11 



Relationship Between Expected Priority Change 
for Academic Progress and Employment Outcomes 
and Student Satisfaction 



lis 



Priority Change in 
Academic Progress 
ana umpioyment 
Outcomes 


Declines 
2 Levels 


Declines 
1 Level 


Priority Change in Student Satisfaction 

No Increases Increases Increases 
Change 1 Level 2 Levels 3 Levels 


Inaeases 
4 Levels 


n 


Lyecunes z Levels 


0.0% 


0.0% 


0.0% 


50.0% 


0.0% 


0.0% 


50.0% 


2 


Declines 1 Level 


0.0 


5.1 

7 A 


51.3 
55 


25.6 


7.7 

/.I 


10.3 
5.9 


0.0 


39 


No Change 


0.0 


4.4 

00.0 


70.6 
66.4 


11.3 
42.4 


4.7 
38.1 


8.1 
41.2 


0.9 
20.0 


345 


Increases 1 Level 


0.1 
100.0 


25.9 


55.8 
22.4 


19.0 
30.4 


8.2 
28.6 


7.5 
16.2 


4.1 
40.0 


14 


Increases 2 Levels 


0.0 


3.3 
7.4 


33.3 
2.2 


20.8 
5.4 


8.3 
4.8 


29.2 
10.3 


0.0 


24 


Increases 3 Levels 


0.0 


0.0 


27.9 
3.3 


18.6 
8.7 


16.3 
16.7 


30.2 
19.1 


7.0 
20.0 


43 


Increases 4 Levels 


0.0 


8.3 

3.7 


8.3 
0.3 


8.3 
1.1 


16.7 
4.8 


41.7 
7.4 


16.7 
13.3 


12 


r = .29 


n = 1 


27 


366 


92 


42 


68 


15 


611 



o 
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Priority Change in Declines 

Student Learning Skills 2 Levels 

Declines 2 Levels 0.0% 

Declines 1 Level 0.0 

No Change 0.0 

Inaeases 1 Level 0.0 

Inaeases 2 Levels 1.6 

100.0 

Increases 3 Levels 0.0 
Increases 4 Levels 0.0 
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Table 12 

Relationship Between Expected Priority Change 
for Student Learning Skills 
and Student Satisfaction 





Priority Change in Student Satisfaction 






Declines 


No 


Increases 


Increases 


Ino'easGs 


Increases 




1 Level 


Chanee 


1 Level 


2 Levels 


3 Levels 


4 Levels 


n 


0.v1% 


0.0% 


0.0% 


0.0% 


0.0% 


100 0% 


1 












6.3 




0.0 


59.1 


18.2 


4.5 


18.2 


0.0 


22 




3.7 


4.5 


2.5 


5.8 








77.8 


10.5 


2.0 


5.6 


0.8 


248 


34.8 


55.0 


29.5 


12.5 


20.3 


12.5 


4.1 


49.7 


21.1 


5.4 


16.3 


3.4 


147 


26.1 


20.8 


35.2 


20.0 


34.8 


31.3 




6.6 


34.4 


23.0 


23.0 


4.9 


6.6 


61 


17.4 


6.0 


15.9 


35.0 


4.3 


25.0 




5.8 


49.3 


13.0 


8.7 


21.7 


1.4 


69 


17.4 


9.7 


12.2 


15.0 


21.7 


6.3 


2.5 


42.5 


10.0 


15.0 


22.5 


7.5 


40 


4.3 


4.8 


4.5 


15.0 


13.0 


18.8 
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Table 13 



Mean Importance Index Score for Current Priority Level 
Associated with Student Outcomes Measures for 
Assessing Institutional Effectiveness 

Current Academic Progress & Student Student 

Priority Level Employment Outcomes Learning Skills Satisfaction 

Highest 74% 87% 87% 

(73) (64) (66) 

High 71 85 86 

(329) (237) (310) 

Not High 65 79 81 

(97) (110) (105) 

None 80 80 83 

(3) (24) (10) 

Not Used 69 83 80 

(73) (136) (121) 

Total 70% 83% 84% 

(584) (571) (612) 



F = 4.45 4.07 6.99 

r = .09 .08 .19 

eta - .17 .17 .21 
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Table 14 



Mean Importance Index Score for Future Priority Level 
Associated with Student Outcomes Measures for 
Assessing Institutional Effectiveness 



Future 

Priority Level 



Academic Progress & 
Employment Outcomes 



Student 
Learnine Skills 



Student 
Satisfaction 



Highest 
High 
Not High 
None 
Total 



74% 
(156) 

70 
(376) 

63 
(55) 

66 
(2) 

70% 
(589) 



86% 
(180) 

83 
(339) 

78 
(69) 

84 
(4) 

83% 
(592) 



87% 
(105) 

85 
(431) 

79 
(82) 

65 
(2) 

84% 
(620) 



F = 6.79 
r = .18 
eta = .18 



5.24 
.15 
.16 



7.07 
.17 
.18 
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Table 15 



Change in Priority 
Associated with Student Outcomes Measures 
and Mean Scores on Importance Index 



Change in Future Academic Progress & Student Student 

Priority Level Employment Outcomes Learning Skills Satisfaction 



Declines 2 Levels 


84% 


O/ /O 


lUUTo 




n = (2) 


(1) 


(1) 


DecUnes 1 Level 


69 




OO 




n = (38) 


(20) 


(28) 


No Change 


71 




OD 




n = (316) 


(237) 


(352) 


increases i Level 


70 


81 


83 




n = (142) 


(144) 


(92) 


Increases 2 Levels 


67 


82 


80 




n = (27) 


(60) 


(43) 


Increases 3 Levels 


68 


83 


81 




n = (42) 


(65) 


(72) 


Increases 4 Levels 


81 


&i 


85 




n = (13) 


(38) 


(17) 


Total 


70% 


83% 


84% 




n = (580) 


(565) 


(605) 



F = 1.81 1.40 2.33 

(sig = .10) (sig = .22) (sig - .03) 

r = .00 .00 -.12 

eta = .14 .12 .15 
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Table 16 



Mean Use Index Score for Current Priority Level 
Associated with Student Outcomes Measures for 
Assessing Ini>titutional Effectiveness 



Current 
Priority Level 

Highest 
High 
Not Hgh 
None 
Not Used 
Total 



Academic Progress & 
Employment Outcomes 

42% 
(85) 

41 
(358) 

35 
(102) 

29 
(4) 

17 
(76) 

37% 
(625) 



Student 
Learning Skills 

38% 
(68) 

29 
(244) 

17 
(116) 

35 
(24) 

54 
(147) 

21% 
(599) 



Student 
Satisfaction 

63% 
(69) 

59 

(322) 

36 

(105) 



(10) 



(120) 

44% 
(626) 



r = 20.03 
r = .3? 
eta = .34 



6.77 
.44 
.45 



0.60 
.55 
.56 
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Table 17 
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Mean Use Score for Future Priority Level 
Associated with Student Outcomes Measures for 
Assessing Institutional Effectiveness 

Current Academic Progress & Student Student 

Priority Lev^l Emplovment Outcomes Learning Skills Satisfaction 

Highest 38% 25% 39% 

(166) (188) (105) 

High 38 19 47 

(402) (352) (445) 

Not High 30 13 32 

(59) (74) (82) 

None 32 05 

(3) (4) (2) 



Total 37% 20% 44% 

(630) (618) (634) 



F = 1.90 (n.s.) 5.39 3.73 

r = .07 .16 .08 



^0 



eta .09 .16 .13 



Table 18 



Change in Priority 
Associated with Student Outcomes Measures and 
Mean Scores on Use Index 



Change in Future Academic Progress & Student Student 

Priority Level Employment Outcomes Learning Skills Satisfaction 





DO to 




1 C\C\Of 

1UU% 




n = (2) 


(1) 


(1) 








/I 




n = (39) 


(22) 


(28) 




41 




Do 




n = (344) 


(246) 


(368) 


Increases 1 Level 


37 


21 


36 




n = (149) 


(148) 


(93) 


Increases 2 Levels 


25 


71 


1 




n = (30) 


(63) 


(40) 


Increases 3 Levels 


16 


46 






n =r (42) 


(73) 


(73) 


Increases 4 Levels 


21 


81 






n = (13) 


(39) 


(16) 


Total 


37% 


21% 


45% 




n = (619) 


(592) 


(619) 



F = 11.11 18.23 42.30 

(sig = .000) (sig = .000) (sig = .000) 

r = -.30 -.37 -.53 

eta = .31 .40 .54 
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Table 19 



Mean Score on Importance Index 
and Enrollment Size of Institution 



Academic Progress & Student Student 

Enrollment Employment Outcomes Learning Skil ls Satisfaction 

1-1000 65% 82% 82% 

(97) (101) (105) 

1001-3500 71 83 85 

(237) (247) (245) 

3501-7000 71 83 85 

(120) (116) (124) 

7001-500000 71 84 84 

(111) (108) (117) 

Total 70% 83% 84% 

(565) (572) (591) 

F = 3.50 0.56 1.77 
(sig = .02) (sig = .64) (sig = .15) 

r = .09 .05 .05 

eta = .14 .05 .09 
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Table 20 





Mean Score on Use Index and Enrolln\ent 






Size of Institution 






Acaaeniic rrogress & 


btudent 


Student 


17nm11monf 

Etl li IJlLll ici 1 1 


Emplovment Outcomes 


Learning Skills 


batisraction 


1-1000 


oo/o 


21% 


37% 




(108) 


(108) 




1001-3500 




on 


46 




(254) 


(254) 




3501-7000 


38 


18 


41 








(127) 


7001-500,000 


DO 


1Q 


40 








(117) 


Total 


36% 


19% 


43% 




(602) 


(602) 


(608) 




F = 0.68 


0.55 


1.62 




(sig = .57) 


(sig = .65) 


(sig = .18) 




r = .04 


-.04 


-.01 




eta = .06 


.05 


.09 



123 



129 



r 



Table 21 



Mean Score on Importance Index 
and Accreditation Region 



Rpcrion 


Academic Progress & 


Student 


Student 




Lt^arrimg oKlllS 


batistaction 


New England 


68% 


85% 


76% 










Middle States 


70 


81 


84 








\77) 


Southern 


71 


83 


85 










North Central 


68 


83 


84 




(221) 


V (218) 


(228) 


Northwest 


73 


90 


86 




(7) 


(6) 


(8) 


Western 


6 


98 


484 




(78) 


(79) 


(82) 


Total 


70% 


83% 


84% 




(605) 


(620) 


(639) 




F = 1.36 


F = .68 


F = 2.74 




(sig = .24) 


(sig = .64) 


(sig = .02) 




eta = .11 


.07 


.15 




124 



130 



Table 22 



Mean Score on Use Index 
and Accreditation Region 





Academic Progress & 


Student 


Student 


Region 


Emplovment Outcomes 




OalloXaLllUIl 


New England 




32% 


11% 


37% 






(28) 


(28) 


(29) 


Middle States 






19 


43 






(80) 




\o\j/ 


Southern 




3A 


25 


51 






(221) 


(221) 


^222) 


North Central 




36 


18 


41 








(235) 


(237) 


Northwest 






10 


40 






(7) 


(7) 


(7) 


Western 




30 


18 


32 






(80) 


(80) 


(82) 


Total 




37% 


20% 


43% 






(651) 


(651) 


(657) 




F = 


= 2.36 


0.55 


3.13 




(sig 


= .04) 


(sig = .65) 


(sig = .01) 




eta 


= .13 


.05 


.15 



125 
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Table 23 



Mean Score in Importance Index 
and Setting of Institution 



Academic Progress & Student Student 

Seeing Employment Outcomes Learning Skills Satisfaction 



Rural 


70% 


82% 


85% 




(195) 


(193) 




City 


72 


82 


85 




(75) 


(79) 


(76) 


Suburban 


69 


84 


84 




(224) 


(231) 


(242) 


Urban 


71 


85 


85 




(109) 


(115) 


(119) 


Total 


70% 


83% 


84% 




(603) 


(618) 


(o37) 




F = 0.92 


1.76 


0.23 




(sig = .43) 


(sig = .15) 


(sig = .88) 




eta = .07 


.09 


.03 



126 



132 



Table 24 



Mean Score on Use Index 
and Setting of Institution 



Academic Progress & Student Student 

Setting Employment Outcomes Learning Skills Satisfaction 

Rural 36% 20% 40% 

(203) (203) (203) 

City 37 21 39 

(78) (78) (80) 

Suburban 37 19 40 

(249) (249) (252) 

Urban 36 22 40 

(119) (119) (120) 

Total 37% * 20% 40% 

(649) (649) (655) 

F = 0.13 0.38 0.66 
(sig = .95) (sig = .77) (sig = .58) 

r = .01 .01 .00 

eta = .02 .04 .06 



ERIC 
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133 



Table 25 



Coordinator of Assessment for Institutional 
Effectiveness and Student Outcomes Measures: 
Mean Importance Scores 



Coordinator 

Both Student 
Outcomes and 
Institutional 
Effectiveness 

Institutional 

Effectiveness 

Only 

No Coordinator 



Total 



Academic Progress 
and Employment 
Outcomes Measures 



71% 
(310) 



68 
(57) 

69 
(238) 

70% 
(605) 



Student 
Learning 
Skills 



83% 
(321) 



83 
(55) 

83 
(244) 

83% 
(620) 



Studen': 
Satisfaction 



86% 
(324) 



82 
(60) 

83 
(255) 

84% 
(639) 



F = 
r = 
eta = 



1.20 
(sig=.30) 
.05 

.06 



0.12 
(sig=.89) 
-.02 

.02 



4.04 
(sig=.02) 
.10 

.11 



128 



134 



Table 26 



Coordinator of Assessment for Institutional 
Effectiveness and Student Outcomes Measures: 
Mean Use Scores 



Academic Progress Student 

and Employment Learning Student 

Coordinator Outcomes Measures Skills Satisfaction 

Both Student 

Outcomes and 

Institutional 41% 25% 52% 

Effectiveness (331) (331) (333) 

Institutional 

Effectiveness 33 15 34 

Only (63) (63) (63) 

No Coordinator 32 13 34 

(257) (257) (261) 

Total 37% 20% 43% 

(651) (651) (657) 



F = 10.61 16.37 18.69 

(sig=.00) (sig==.00) (sig-.OO) 

r = .17 -.21 .22 

eta = .18 .22 .23 



m 



Table 27 



Priority Currently Associated with Use of 
Student Outcomes Measures by Coordinator of Assessment: 
Academic Progress and Employment Outcomes Measures 

_ . . , , Institutional No 

Priority Level goth Effectiveness Only Coordinator 

Highest 17.1% 11.1% 8.9% 

High 60.1 57.1 52.8 

Not High 15.0 17.5 18.5 

None 0.6 o.O 0 8 

Not Used 7.2 14.3 I9*c 

n = 333 63 248 

r = .20 

eta, priority dependent = .20 
Cramer's v = .14 



Table 28 

Priority Currently Associated with Use of 
Student Outcomes Measures by Coordinator of Assessment: 
Student Learning Skills Outcomes Measures 

. . Institutional No 

Priority Level Both Effectiveness Only Coordinator 

Highest 12.1% 8.3% 10.5% 

High 45.8 33.3 37.0 

Not High 19.0 20.0 18 9 

None 3.4 11.7 2.9 

Not Used 19.6 26.7 



r = .12 

eta, priority dependent - .13 
Cramer's v = .13 



30.7 



n « 321 60 238 
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Table 29 



Priority Currently Associated with Use of Student 
Outcomes Measures by Coordinator of Assessment: 







Institutional 


No 


Priority Level 


Both 


Effectiveness Only 


Coordinator 


Highest 


13.3% 


5.3% 


9.5% 


High 


56.0 


47.4 


44.8 


Not High 


16.9 


22.8 


16.3 


None 


0.6 


3.5 


2.4 


Not Used 


13.3 


21.1 


27.0 




n = 332 


57 


252 



r = .18 

eta, priority dependent = .18 
Cramer's v = .15 



Table 30 



Future Priority to be Associated with Use of 
Student Outcomes Measures by Coordinator of Assessment: 
Academic Progress and Employment Outcomes Measures 







Institutional 


No 


Prioritv Level 


Both 


Effectiveness Onlv 


Coordinator 


Highest 


28.1% 


19.4% 


24.8% 


High 


62.7 


69.4 


65.0 


Not High 


8.7 


9.7 


10.2 


None 


0.6 


1.6 


0.0 




n = 335 


62 


254 



(not significant) 
r = .03 

eta, priority dependent = .06 
Cramer's v = .07 
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Table 31 

Future Priority to be Associated with Use of 
Student Outcomes Measures by Coordinator of Assessment: 
Student Learning Skills Outcomes Measures 



Institutional No 

I^iprity I^y^l Bgth Effecriveness Only Coordinator 

Highest 33.7% 24.6% 28.1% 

™gh 56.7 59.0 56.9 

Not High 9.5 14.8 13 8 

None 0.0 1.6 ^2 

n = 326 61 253 



(not significant) 
r = .09 

eta, priority dependent = .11 
Cramer's v = .09 



Table 32 

Future Priority to be Associated with Use of 
Student Outcomes Measures by Coordinator of Assessment: 

Student Satisfaction 



. . Institutional No 

Priority Le^ Both Effectiveness Only Coordinator 

13.6% 14.1% 

High 68.7 62.7 72 5 

Not High 11.6 20.3 133 

None 0.0 3.4 0.0 

n = 335 59 255 



r « .07 

eta, priority dependent = .16 
Cramer's v - .14 



132 



135 



Table 33 



Satisfaction w: ih Use of Student Outcomes 
Measures by Coordinator of Assessment: 
Academic Progress and Employment Outcomes Measures 

Institutional No 
Satisfaction Level Both Effectiveness Only Coordinator 



Very High 4.6% 3.4% 2.5% 

High 32.8 18.6 19.8 

Medium 46.8 61.0 45.7 

Low 13.7 16.9 26.7 

Very Low 2.1 0.0 5.3 

n = 329 59 243 

r = .20 

eta; satisfaction dependent = .20 
Cramer's v - .67 



Table 34 

Satisfaction with Use of Student Outcomes 
Measures by Coordinator of Assessment: 
Student Learning Skills 







Institutional 


Satisfaction Level 


Poth 


Effectiveness Onlv 


Very High 


1.3% 


0.0% 


High 


21.4 


13.2 


Medium 


40.5 


39.6 


Low 


27.2 


32.1 


Very Low 


9.7 


15.1 



No 

Coordinator 

0.8% 
12.7 
35.0 
40.1 
11.4 



n = 309 



53 



237 



r = .14 

eta, satisfaction dependent - .14 
Cramer's v = .12 
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Table 35 



Satisfaction with Use of Student Outcomes 
Measures by Coordinator of Assessment; 
Student Satisfaction 



Satisfaction Level 

Very High 
High 
Medium 
Low 

Very Low 



Both 

4.0% 
38.5 
40.1 
13.4 

4.0 



n = 322 

r = .20 

eta, satisfaction dependent = .21 
Cramer's v = .16 



Institutional 
Effectiveness Only 

1.8% 
26.8 
35.7 
28.6 

7.1 

56 



No 
Coordinator 

2.5% 
22.2 
43.1 
23.8 

8.4 

239 



Rating of Effort 

Excellent 
Good 

Neutral/Neither 
Poor 

Very Poor 



Table 36 

rail Rating of Institutional Effort in Assessment 
by Coordinator of Assessment: 



Both 

11.7% 
55.9 
18.6 
13.5 
0.3 



n = 333 

r = .38 

eta, satisfaction dependent = .38 
Cramer's v = .53 



Institutional 
Effectiveness Only 

1.8% 
43.9 
35.1 
15.8 

3.5 

57 



No 

Coordinator 

2.1% 
26.6 
34.9 
27.8 

8.7 

241 
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Table 37 

Overall Rating of Effectiveness 
of Institutional Assessment Effort 
by Coordinator of Assessment: 



Effectiveness Rating 

Very Effective 
Moderately Effective 
Average 

Moderately Ineffective 
Very Ineffective 





Institutional 


No 


Both 


Effectiveness Onlv 


Coordinator 


7.1% 


3.6% 


0.9% 


32.9 


18.2 


24.3 


43.5 


45.5 


38.7 


14.9 


29.1 


26.5 


1.6 


3.6 


9.6 


n = 322 


55 


230 



r = .25 

eta, satisfaction dependent = .26 
Cramer's v = .33 
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